Skip to content

Commit 83cba25

Browse files
niyueniyue
authored
GH-37848: [C++][Gandiva] Migrate LLVM JIT engine from MCJIT to ORC v2/LLJIT (#39098)
### Rationale for this change Gandiva currently employs MCJIT as its internal JIT engine. However, LLVM has introduced a newer JIT API known as ORC v2/LLJIT since LLVM 7.0, and it has several advantage over MCJIT, in particular, MCJIT is not actively maintained, and is slated for eventual deprecation and removal. ### What changes are included in this PR? * This PR replaces the MCJIT JIT engine with the ORC v2 engine, using the `LLJIT` API. * This PR adds a new JIT linker option `JITLink` (https://llvm.org/docs/JITLink.html), which can be used together with `LLJIT`, for LLVM 14+ on Linux/macOS platform. It is turned off by default but could be turned on with environment variable `GANDIVA_USE_JIT_LINK` ### Are these changes tested? Yes, they are covered by existing unit tests ### Are there any user-facing changes? * `Configuration` class has a new option called `dump_ir`. If users would like to call `DumpIR` API of `Projector` and `Filter`, they have to set the `dump_ir` option first. * Closes: #37848 Authored-by: Yue Ni <niyue.com@gmail.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
1 parent ccc674c commit 83cba25

18 files changed

Lines changed: 441 additions & 199 deletions

‎cpp/cmake_modules/FindLLVMAlt.cmake‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -93,8 +93,8 @@ if(LLVM_FOUND)
9393
debuginfodwarf
9494
ipo
9595
linker
96-
mcjit
9796
native
97+
orcjit
9898
target)
9999
if(LLVM_VERSION_MAJOR GREATER_EQUAL 14)
100100
list(APPEND LLVM_TARGET_COMPONENTS passes)

‎cpp/src/gandiva/configuration.h‎

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -37,10 +37,12 @@ class GANDIVA_EXPORT Configuration {
3737

3838
explicit Configuration(bool optimize,
3939
std::shared_ptr<FunctionRegistry> function_registry =
40-
gandiva::default_function_registry())
40+
gandiva::default_function_registry(),
41+
bool dump_ir = false)
4142
: optimize_(optimize),
4243
target_host_cpu_(true),
43-
function_registry_(function_registry) {}
44+
function_registry_(std::move(function_registry)),
45+
dump_ir_(dump_ir) {}
4446

4547
Configuration() : Configuration(true) {}
4648

@@ -50,11 +52,13 @@ class GANDIVA_EXPORT Configuration {
5052

5153
bool optimize() const { return optimize_; }
5254
bool target_host_cpu() const { return target_host_cpu_; }
55+
bool dump_ir() const { return dump_ir_; }
5356
std::shared_ptr<FunctionRegistry> function_registry() const {
5457
return function_registry_;
5558
}
5659

5760
void set_optimize(bool optimize) { optimize_ = optimize; }
61+
void set_dump_ir(bool dump_ir) { dump_ir_ = dump_ir; }
5862
void target_host_cpu(bool target_host_cpu) { target_host_cpu_ = target_host_cpu; }
5963
void set_function_registry(std::shared_ptr<FunctionRegistry> function_registry) {
6064
function_registry_ = std::move(function_registry);
@@ -65,6 +69,9 @@ class GANDIVA_EXPORT Configuration {
6569
bool target_host_cpu_; /* set the mcpu flag to host cpu while compiling llvm ir */
6670
std::shared_ptr<FunctionRegistry>
6771
function_registry_; /* function registry that may contain external functions */
72+
// flag indicating if IR dumping is needed, defaults to false, and turning it on will
73+
// negatively affect performance
74+
bool dump_ir_ = false;
6875
};
6976

7077
/// \brief configuration builder for gandiva
@@ -83,6 +90,12 @@ class GANDIVA_EXPORT ConfigurationBuilder {
8390
return configuration;
8491
}
8592

93+
std::shared_ptr<Configuration> build_with_ir_dumping(bool dump_ir) {
94+
std::shared_ptr<Configuration> configuration(
95+
new Configuration(true, gandiva::default_function_registry(), dump_ir));
96+
return configuration;
97+
}
98+
8699
std::shared_ptr<Configuration> build(
87100
std::shared_ptr<FunctionRegistry> function_registry) {
88101
std::shared_ptr<Configuration> configuration(

0 commit comments

Comments
 (0)