Skip to content

Enable JIT compilation on macOS#100947

Open
alexey-milovidov wants to merge 9 commits intomasterfrom
enable-jit-on-macos
Open

Enable JIT compilation on macOS#100947
alexey-milovidov wants to merge 9 commits intomasterfrom
enable-jit-on-macos

Conversation

@alexey-milovidov
Copy link
Copy Markdown
Member

The APPLE condition was disabling the embedded compiler (JIT) and DWARF parser on macOS. The comment only explains the MSAN exclusion, and the Apple exclusion appears to be a leftover from early days when LLVM didn't build cleanly on macOS. Remove it to enable JIT on macOS by default.

Changelog category (leave one):

  • Improvement

Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):

Enable JIT compilation on macOS.

Documentation entry for user-facing changes

  • Documentation is written (mandatory for new features)

The `APPLE` condition was disabling the embedded compiler (JIT) and DWARF parser on macOS. The comment only explains the MSAN exclusion, and the Apple exclusion appears to be a leftover from early days when LLVM didn't build cleanly on macOS. Remove it to enable JIT on macOS by default.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@clickhouse-gh
Copy link
Copy Markdown
Contributor

clickhouse-gh bot commented Mar 27, 2026

Workflow [PR], commit [90251dc]

Summary:

job_name test_name status info comment
Stress test (amd_debug) failure
Logical error: Shard number is greater than shard count: shard_num=A shard_count=B cluster=C (STID: 5066-457d) FAIL cidb
Stress test (arm_tsan) failure
Hung check failed, possible deadlock found FAIL cidb, issue
Stress test (arm_msan) failure
MemorySanitizer: use-of-uninitialized-value (STID: 2410-47e0) FAIL cidb, issue
Upgrade check (amd_release) failure
Error message in clickhouse-server.log (see upgrade_error_messages.txt) FAIL cidb
Integration tests (amd_asan_ubsan, db disk, old analyzer, 1/6) error
Integration tests (amd_asan_ubsan, db disk, old analyzer, 3/6) error
Integration tests (amd_asan_ubsan, db disk, old analyzer, 5/6) error
Integration tests (amd_asan_ubsan, db disk, old analyzer, 6/6) error
Integration tests (amd_binary, 2/5) error
Integration tests (amd_binary, 3/5) error

AI Review

Summary

This PR removes the legacy APPLE guard that disabled ENABLE_EMBEDDED_COMPILER/ENABLE_DWARF_PARSER defaults, so JIT is enabled on macOS by default, while keeping SANITIZE=memory disabled as before. It also adds an explicit HAVE_LIBRT=0 override for Apple targets to avoid incorrect librt linkage when LLVM is configured. The change is small, targeted, and consistent with the stated motivation.

ClickHouse Rules
Item Status Notes
Deletion logging
Serialization versioning
Core-area scrutiny
No test removal
Experimental gate
No magic constants
Backward compatibility
SettingsChangesHistory.cpp
PR metadata quality
Safe rollout
Compilation time
Final Verdict
  • Status: ✅ Approve

@clickhouse-gh clickhouse-gh bot added pr-improvement Pull request with some product improvements submodule changed At least one submodule changed in this PR. labels Mar 27, 2026
alexey-milovidov and others added 4 commits March 27, 2026 22:13
When cross-compiling to macOS, LLVM's `config-ix.cmake` detects
`HAVE_LIBRT` from the Linux build host, but `librt` doesn't exist
on macOS. The LLVM Orc JIT libraries (`LLVMOrcTargetProcess`,
`LLVMOrcJIT`, `LLVMOrcDebugging`) link `rt` when `HAVE_LIBRT` is
set, causing: `ld: library not found for -lrt`.

Fix by setting `HAVE_LIBRT` to false for Apple targets before
entering the LLVM subdirectory.

https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=100947&sha=98ea2118ba286f3973765cc477b882e5ecd12080&name_0=PR&name_1=Build%20%28amd_darwin%29

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
alexey-milovidov added a commit that referenced this pull request Mar 29, 2026
…es than `skip_first_lines`

When `input_format_csv_skip_first_lines` or `input_format_tsv_skip_first_lines`
is set to a value larger than the number of lines in the file,
`skipPrefixBeforeHeader` enters an infinite loop doing `pread` at EOF.
Each call to `readRow` succeeds with an empty result even at EOF, so the
loop never terminates.

Add an EOF check before each `readRow` call to break out of the loop
when the file is exhausted.

Found via AST fuzzer in Stress test (amd_msan) on PR #100947.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown
Member

@Algunenano Algunenano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to work fine:

:)         SELECT
            COUNT()
        FROM
            jit_test
        WHERE
            NOT ignore(a / b + c / d + e / f + g / h + i / j)
        SETTINGS
            compile_expressions = 0;

SELECT COUNT()
FROM jit_test
WHERE NOT ignore(((((a / b) + (c / d)) + (e / f)) + (g / h)) + (i / j))
SETTINGS compile_expressions = 0

Query id: 4d9b1028-d3aa-483b-a93a-2c682b905323

   ┌───COUNT()─┐
1. │ 200000000 │ -- 200.00 million
   └───────────┘

1 row in set. Elapsed: 1.774 sec. Processed 190.86 million rows, 15.27 GB (107.58 million rows/s., 8.61 GB/s.)
Peak memory usage: 294.53 KiB.

:)         SELECT
            COUNT()
        FROM
            jit_test
        WHERE
            NOT ignore(a / b + c / d + e / f + g / h + i / j)
        SETTINGS
            compile_expressions = 1,
            min_count_to_compile_expression = 1

SELECT COUNT()
FROM jit_test
WHERE NOT ignore(((((a / b) + (c / d)) + (e / f)) + (g / h)) + (i / j))
SETTINGS compile_expressions = 1, min_count_to_compile_expression = 1

Query id: e05cbfe2-85d5-4a6f-85af-3f8c9315eb23

   ┌───COUNT()─┐
1. │ 200000000 │ -- 200.00 million
   └───────────┘

1 row in set. Elapsed: 0.211 sec. Processed 200.00 million rows, 16.00 GB (949.12 million rows/s., 75.93 GB/s.)
Peak memory usage: 295.34 KiB.

@Algunenano Algunenano self-assigned this Mar 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-improvement Pull request with some product improvements submodule changed At least one submodule changed in this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants