Skip to content

codegen: LLVM 21 ThreadSafeContext and EHFrame compatibility#61273

Merged
oscardssmith merged 2 commits intomasterfrom
gb/llvm21-threadsafe-context
Mar 11, 2026
Merged

codegen: LLVM 21 ThreadSafeContext and EHFrame compatibility#61273
oscardssmith merged 2 commits intomasterfrom
gb/llvm21-threadsafe-context

Conversation

@gbaraldi
Copy link
Copy Markdown
Member

@gbaraldi gbaraldi commented Mar 9, 2026 •
edited
Loading

Summary

  • Add withContextDo compatibility shim for ThreadSafeContext API changes in LLVM 21 (which removes getLock(), Lock, and getContext() in favor of callback-based withContextDo)
  • Replace all manual lock/unlock patterns with scoped withContextDo callbacks across jitlayers.h, jitlayers.cpp, aotcompile.cpp, and disasm.cpp
  • Add JLEHFrameRegistrationPlugin for LLVM 21+ (which removes jitlink::EHFrameRegistrar), using JITLink allocation actions for EH frame registration
  • Debug asserts ensure get_context()/get_module() are only called inside withContextDo scope
  • Lock hierarchy respected: TSC locks released before addOutput()/R->replace() which can trigger materialization

Supersedes #59946 and #60358, forward-porting those changes to current master.

Test plan

  • Builds and passes smoke test on LLVM 20
  • Builds and passes smoke test on LLVM 21 (tested with libLLVM 21.1.2 JLL)
  • CI

This pull request was written with the assistance of generative AI (Claude).

🤖 Generated with Claude Code

There is quite a few whitespace changes so ignoring it is quite helpful

@gbaraldi gbaraldi force-pushed the gb/llvm21-threadsafe-context branch from 8f8c746 to a445648 Compare March 9, 2026 18:15
@gbaraldi gbaraldi requested a review from xal-0 March 9, 2026 18:28
@giordano giordano added compiler:codegen Generation of LLVM IR and native code compiler:llvm For issues that relate to LLVM labels Mar 9, 2026
@gbaraldi gbaraldi added the don't squash Don't squash merge label Mar 9, 2026
@giordano
Copy link
Copy Markdown
Member

giordano commented Mar 9, 2026

Analyzegc is unhappy: https://buildkite.com/julialang/julia-master/builds/55503#019cd3d0-ac7c-40b7-99ec-70394e193ab7/L2196

/cache/build/builder-amdci5-5/julialang/julia-master/src/jitlayers.cpp:984:9: error: Safepoints disabled at end of function [julia.GCChecker]
  984 |         F->setName(*Sym);
      |         ^
[...]
/cache/build/builder-amdci5-5/julialang/julia-master/src/jitlayers.cpp:2413:12: error: Calling potential safepoint as SimpleFunctionCall from function annotated JL_NOTSAFEPOINT [julia.GCChecker]
 2413 |     return withContextDo(TSM.getContext(), [&](LLVMContext*) -> std::unique_ptr<MemoryBuffer> {
      |            ^

@gbaraldi gbaraldi force-pushed the gb/llvm21-threadsafe-context branch from a445648 to c0284d5 Compare March 9, 2026 19:52
@xal-0
Copy link
Copy Markdown
Member

xal-0 commented Mar 9, 2026 •
edited
Loading

It's awkward that jl_codegen_output_t owns a ThreadSafeModule to begin with, especially since we have usually unique ownership of the llvm::Module and LLVMContext. I've got a branch that refactors it to hold a reference to the module, so we can just create the module and context directly in the typical case and .withModuleDo() to emit to a user-provided ThreadSafeModule. Could you drop c0284d5 and take 27715b2 from my branch instead?

@gbaraldi gbaraldi force-pushed the gb/llvm21-threadsafe-context branch 2 times, most recently from e0d3627 to 86b0ff5 Compare March 10, 2026 15:17
gbaraldi and others added 2 commits March 10, 2026 17:45
@gbaraldi @yuyichao @xal-0
codegen: Add LLVM 21 EHFrameRegistrationPlugin alloc actions
f6cf3eb 
LLVM 21 removes `jitlink::EHFrameRegistrar` and changes the
`EHFrameRegistrationPlugin` constructor to use a factory method.  Use the new
EHFrameRegistrationPlugin constructor to pass our register/deregister actions.

Supersedes #60358.

Co-Authored-By: yuyichao <yyc1992@gmail.com>
Co-Authored-By: Sam Schweigel <sam.schweigel@juliahub.com>
@xal-0
Take references to Module and LLVMContext in jl_codegen_output_t
52bc353
@xal-0 xal-0 force-pushed the gb/llvm21-threadsafe-context branch from 86b0ff5 to 52bc353 Compare March 11, 2026 00:47
@xal-0
Copy link
Copy Markdown
Member

xal-0 commented Mar 11, 2026

I deleted JLEHFrameRegistrationPlugin; we can use the upstream EHFrameRegistrationPlugin and pass our alloc actions directly.

@giordano
Copy link
Copy Markdown
Member

giordano commented Mar 11, 2026 •
edited
Loading

All green. Is this ready?

@gbaraldi
Copy link
Copy Markdown
Member Author

gbaraldi commented Mar 11, 2026

I think so thanks @xal-0 for pushing it forward

@oscardssmith oscardssmith merged commit f59c6df into master Mar 11, 2026
8 checks passed
@oscardssmith oscardssmith deleted the gb/llvm21-threadsafe-context branch March 11, 2026 13:44
@oscardssmith
Copy link
Copy Markdown
Member

oscardssmith commented Mar 11, 2026

Accidentally squashed. Sorry. Can we make it so that the dont squash label actually prevents squashing? The big green button is way too easy to press.

@giordano giordano mentioned this pull request Mar 11, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@xal-0 xal-0 xal-0 approved these changes

Assignees

No one assigned

Labels

compiler:codegen Generation of LLVM IR and native code compiler:llvm For issues that relate to LLVM don't squash Don't squash merge

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@gbaraldi @giordano @xal-0 @oscardssmith