Closed
Conversation
the custom allocators are needed for Sparse Arrays
adienes
pushed a commit
that referenced
this pull request
Feb 12, 2025
At some point LLVM on MacOS started doing frame pointer optimization by default. We should ask for a frame pointer on every function, on all platforms. Prior to this change, on `1.11.3+0.aarch64.apple.darwin14`: ``` julia> @code_native ((x,y) -> Core.Intrinsics.add_float(x,y))(1.0,2.0) .section __TEXT,__text,regular,pure_instructions .build_version macos, 15, 0 .globl "_julia_#1_678" ; -- Begin function julia_#1_678 .p2align 2 "_julia_#1_678": ; @"julia_#1_678" ; Function Signature: var"#1"(Float64, Float64) ; ┌ @ REPL[1]:1 within `#1` ; %bb.0: ; %top ; │ @ REPL[1] within `#1` ;DEBUG_VALUE: #1:x <- $d0 ;DEBUG_VALUE: #1:x <- $d0 ;DEBUG_VALUE: #1:y <- $d1 ;DEBUG_VALUE: #1:y <- $d1 ; │ @ REPL[1]:1 within `#1` fadd d0, d0, d1 ret ; └ ; -- End function .section __DATA,__const .p2align 3, 0x0 ; @"+Core.Float64#680" "l_+Core.Float64#680": .quad "l_+Core.Float64#680.jit" .set "l_+Core.Float64#680.jit", 5490712608 .subsections_via_symbols ``` Prior to this change, on `1.11.3+0.aarch64.linux.gnu`: ``` julia> @code_native ((x,y) -> Core.Intrinsics.add_float(x,y))(1.0,2.0) .text .file "#1" .globl "julia_#1_656" // -- Begin function julia_#1_656 .p2align 2 .type "julia_#1_656",@function "julia_#1_656": // @"julia_#1_656" ; Function Signature: var"#1"(Float64, Float64) ; ┌ @ REPL[1]:1 within `#1` // %bb.0: // %top ; │ @ REPL[1] within `#1` //DEBUG_VALUE: #1:x <- $d0 //DEBUG_VALUE: #1:x <- $d0 //DEBUG_VALUE: #1:y <- $d1 //DEBUG_VALUE: #1:y <- $d1 stp x29, x30, [sp, #-16]! // 16-byte Folded Spill mov x29, sp ; │ @ REPL[1]:1 within `#1` fadd d0, d0, d1 ldp x29, x30, [sp], JuliaLang#16 // 16-byte Folded Reload ret .Lfunc_end0: .size "julia_#1_656", .Lfunc_end0-"julia_#1_656" ; └ // -- End function .type ".L+Core.Float64#658",@object // @"+Core.Float64#658" .section .rodata,"a",@progbits .p2align 3, 0x0 ".L+Core.Float64#658": .xword ".L+Core.Float64#658.jit" .size ".L+Core.Float64#658", 8 .set ".L+Core.Float64#658.jit", 278205186835760 .size ".L+Core.Float64#658.jit", 8 .section ".note.GNU-stack","",@progbits ```
adienes
pushed a commit
that referenced
this pull request
Feb 26, 2025
…aLang#55600) As an application of JuliaLang#55545, this commit avoids the insertion of `:throw_undef_if_not` nodes when the defined-ness of a slot is guaranteed by abstract interpretation. ```julia julia> function isdefined_nothrow(c, x) local val if c val = x end if @isdefined val return val end return zero(Int) end; julia> @code_typed isdefined_nothrow(true, 42) ``` ```diff diff --git a/old b/new index c4980a5c9c..3d1d6d30f0 100644 --- a/old +++ b/new @@ -4,7 +4,6 @@ CodeInfo( 3 ┄ %3 = φ (#2 => x, #1 => #undef)::Int64 │ %4 = φ (#2 => true, #1 => false)::Bool └── goto #5 if not %4 -4 ─ $(Expr(:throw_undef_if_not, :val, :(%4)))::Any -└── return %3 +4 ─ return %3 5 ─ return 0 ) => Int64 ```
adienes
pushed a commit
that referenced
this pull request
Feb 26, 2025
Prior to this, especially on macOS, the gc-safepoint here would cause the process to segfault as we had already freed the current_task state. Rearrange this code so that the GC interactions (except for the atomic store to current_task) are all handled before entering GC safe, and then signaling the thread is deleted (via setting current_task = NULL, published by jl_unlock_profile_wr to other threads) is last. ``` ERROR: Exception handler triggered on unmanaged thread. Process 53827 stopped * thread #5, stop reason = EXC_BAD_ACCESS (code=2, address=0x100018008) frame #0: 0x0000000100b74344 libjulia-internal.1.12.0.dylib`jl_delete_thread [inlined] jl_gc_state_set(ptls=0x000000011f8b3200, state='\x02', old_state=<unavailable>) at julia_threads.h:272:9 [opt] 269 assert(old_state != JL_GC_CONCURRENT_COLLECTOR_THREAD); 270 jl_atomic_store_release(&ptls->gc_state, state); 271 if (state == JL_GC_STATE_UNSAFE || old_state == JL_GC_STATE_UNSAFE) -> 272 jl_gc_safepoint_(ptls); 273 return old_state; 274 } 275 STATIC_INLINE int8_t jl_gc_state_save_and_set(jl_ptls_t ptls, Target 0: (julia) stopped. (lldb) up frame #1: 0x0000000100b74320 libjulia-internal.1.12.0.dylib`jl_delete_thread [inlined] jl_gc_state_save_and_set(ptls=0x000000011f8b3200, state='\x02') at julia_threads.h:278:12 [opt] 275 STATIC_INLINE int8_t jl_gc_state_save_and_set(jl_ptls_t ptls, 276 int8_t state) 277 { -> 278 return jl_gc_state_set(ptls, state, jl_atomic_load_relaxed(&ptls->gc_state)); 279 } 280 #ifdef __clang_gcanalyzer__ 281 // these might not be a safepoint (if they are no-op safe=>safe transitions), but we have to assume it could be (statically) (lldb) frame #2: 0x0000000100b7431c libjulia-internal.1.12.0.dylib`jl_delete_thread(value=0x000000011f8b3200) at threading.c:537:11 [opt] 534 ptls->root_task = NULL; 535 jl_free_thread_gc_state(ptls); 536 // then park in safe-region -> 537 (void)jl_gc_safe_enter(ptls); 538 } ``` (test incorporated into JuliaLang#55793)
adienes
pushed a commit
that referenced
this pull request
Feb 26, 2025
Rebase and extension of @alexfanqi's initial work on porting Julia to RISC-V. Requires LLVM 19. Tested on a VisionFive2, built with: ```make MARCH := rv64gc_zba_zbb MCPU := sifive-u74 USE_BINARYBUILDER:=0 DEPS_GIT = llvm override LLVM_VER=19.1.1 override LLVM_BRANCH=julia-release/19.x override LLVM_SHA1=julia-release/19.x ``` ```julia-repl ❯ ./julia _ _ _ _(_)_ | Documentation: https://docs.julialang.org (_) | (_) (_) | _ _ _| |_ __ _ | Type "?" for help, "]?" for Pkg help. | | | | | | |/ _` | | | | |_| | | | (_| | | Version 1.12.0-DEV.1374 (2024-10-14) _/ |\__'_|_|_|\__'_| | riscv/25092a3982* (fork: 1 commits, 0 days) |__/ | julia> versioninfo(; verbose=true) Julia Version 1.12.0-DEV.1374 Commit 25092a3* (2024-10-14 09:57 UTC) Platform Info: OS: Linux (riscv64-unknown-linux-gnu) uname: Linux 6.11.3-1-riscv64 #1 SMP Debian 6.11.3-1 (2024-10-10) riscv64 unknown CPU: unknown: speed user nice sys idle irq #1 1500 MHz 922 s 0 s 265 s 160953 s 0 s #2 1500 MHz 457 s 0 s 280 s 161521 s 0 s #3 1500 MHz 452 s 0 s 270 s 160911 s 0 s #4 1500 MHz 638 s 15 s 301 s 161340 s 0 s Memory: 7.760246276855469 GB (7474.08203125 MB free) Uptime: 16260.13 sec Load Avg: 0.25 0.23 0.1 WORD_SIZE: 64 LLVM: libLLVM-19.1.1 (ORCJIT, sifive-u74) Threads: 1 default, 0 interactive, 1 GC (on 4 virtual cores) Environment: HOME = /home/tim PATH = /home/tim/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/games TERM = xterm-256color julia> ccall(:jl_dump_host_cpu, Nothing, ()) CPU: sifive-u74 Features: +zbb,+d,+i,+f,+c,+a,+zba,+m,-zvbc,-zksed,-zvfhmin,-zbkc,-zkne,-zksh,-zfh,-zfhmin,-zknh,-v,-zihintpause,-zicboz,-zbs,-zvknha,-zvksed,-zfa,-ztso,-zbc,-zvknhb,-zihintntl,-zknd,-zvbb,-zbkx,-zkt,-zvkt,-zicond,-zvksh,-zvfh,-zvkg,-zvkb,-zbkb,-zvkned julia> @code_native debuginfo=:none 1+2. .text .attribute 4, 16 .attribute 5, "rv64i2p1_m2p0_a2p1_f2p2_d2p2_c2p0_zicsr2p0_zifencei2p0_zmmul1p0_zba1p0_zbb1p0" .file "+" .globl "julia_+_3003" .p2align 1 .type "julia_+_3003",@function "julia_+_3003": addi sp, sp, -16 sd ra, 8(sp) sd s0, 0(sp) addi s0, sp, 16 fcvt.d.l fa5, a0 ld ra, 8(sp) ld s0, 0(sp) fadd.d fa0, fa5, fa0 addi sp, sp, 16 ret .Lfunc_end0: .size "julia_+_3003", .Lfunc_end0-"julia_+_3003" .type ".L+Core.Float64#3005",@object .section .data.rel.ro,"aw",@progbits .p2align 3, 0x0 ".L+Core.Float64#3005": .quad ".L+Core.Float64#3005.jit" .size ".L+Core.Float64#3005", 8 .set ".L+Core.Float64#3005.jit", 272467692544 .size ".L+Core.Float64#3005.jit", 8 .section ".note.GNU-stack","",@progbits ``` Lots of bugs guaranteed, but with this we at least have a functional build and REPL for further development by whoever is interested. Also requires Linux 6.4+, since the fallback processor detection used here relies on LLVM's `sys::getHostCPUFeatures`, which for RISC-V is implemented using hwprobe introduced in 6.4. We could probably add a fallback that parses `/proc/cpuinfo`, either by building a CPU database much like how we've done for AArch64, or by parsing the actual ISA string contained there. That would probably also be a good place to add support for profiles, which are supposedly the way forward to package RISC-V binaries. That can happen in follow-up PRs though. For now, on older kernels, use the `-C` arg to Julia to specify an ISA. Co-authored-by: Alex Fan <alex.fan.q@gmail.com>
adienes
pushed a commit
that referenced
this pull request
Feb 26, 2025
…uliaLang#56300) The pipeline-prints test currently fails when running on an aarch64-macos device: ``` /Users/tim/Julia/src/julia/test/llvmpasses/pipeline-prints.ll:309:23: error: AFTERVECTORIZATION: expected string not found in input ; AFTERVECTORIZATION: vector.body ^ <stdin>:2:40: note: scanning from here ; *** IR Dump Before AfterVectorizationMarkerPass on julia_f_199 *** ^ <stdin>:47:27: note: possible intended match here ; *** IR Dump Before AfterVectorizationMarkerPass on jfptr_f_200 *** ^ Input file: <stdin> Check file: /Users/tim/Julia/src/julia/test/llvmpasses/pipeline-prints.ll -dump-input=help explains the following input dump. Input was: <<<<<< 1: opt: WARNING: failed to create target machine for 'x86_64-unknown-linux-gnu': unable to get target for 'x86_64-unknown-linux-gnu', see --version and --triple. 2: ; *** IR Dump Before AfterVectorizationMarkerPass on julia_f_199 *** check:309'0 X~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ error: no match found 3: define i64 @julia_f_199(ptr addrspace(10) noundef nonnull align 16 dereferenceable(40) %0) #0 !dbg !4 { check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 4: top: check:309'0 ~~~~~ 5: %1 = call ptr @julia.get_pgcstack() check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 6: %ptls_field = getelementptr inbounds ptr, ptr %1, i64 2 check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 7: %ptls_load45 = load ptr, ptr %ptls_field, align 8, !tbaa !8 check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ . . . 42: check:309'0 ~ 43: L41: ; preds = %L41.loopexit, %L17, %top check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 44: %value_phi10 = phi i64 [ 0, %top ], [ %7, %L17 ], [ %.lcssa, %L41.loopexit ] check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 45: ret i64 %value_phi10, !dbg !52 check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 46: } check:309'0 ~~ 47: ; *** IR Dump Before AfterVectorizationMarkerPass on jfptr_f_200 *** check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ check:309'1 ? possible intended match 48: ; Function Attrs: noinline optnone check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 49: define nonnull ptr addrspace(10) @jfptr_f_200(ptr addrspace(10) %0, ptr noalias nocapture noundef readonly %1, i32 %2) #1 { check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 50: top: check:309'0 ~~~~~ 51: %3 = call ptr @julia.get_pgcstack() check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 52: %4 = getelementptr inbounds ptr addrspace(10), ptr %1, i32 0 check:309'0 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ . . . >>>>>> -- ******************** Failed Tests (1): Julia :: pipeline-prints.ll ``` The problem is that these tests assume x86_64, which fails because the target isn't available, so it presumably uses the native target which has different vectorization characteristics: ``` ❯ ./usr/tools/opt --load-pass-plugin=libjulia-codegen.dylib -passes='julia' --print-before=AfterVectorization -o /dev/null ../../test/llvmpasses/pipeline-prints.ll ./usr/tools/opt: WARNING: failed to create target machine for 'x86_64-unknown-linux-gnu': unable to get target for 'x86_64-unknown-linux-gnu', see --version and --triple. ``` There's other tests that assume this (e.g. the `fma` cpufeatures one), but they don't fail, so I've left them as is.
adienes
pushed a commit
that referenced
this pull request
May 11, 2025
Before:
```julia-repl
julia> missing(1)
ERROR: MethodError: objects of type Missing are not callable.
In case you did not try calling it explicitly, check if a Missing has been passed as an argument to a method that expects a callable instead.
The object of type `Missing` exists, but no method is defined for this combination of argument types when trying to treat it as a callable object.┌ Error: Hint-handler nonsetable_type_hint_handler for MethodError in Base caused an error
│ exception =
│ 1-element ExceptionStack:
│ TypeError: non-boolean (Missing) used in boolean context
│ Stacktrace:
│ [1] nonsetable_type_hint_handler(io::Any, ex::Any, arg_types::Any, kwargs::Any)
│ @ Base ./errorshow.jl:1054
│ [2] show_error_hints(::IOContext{Base.TTY}, ::MethodError, ::Vector{Any}, ::Vararg{Vector{Any}})
│ @ Base.Experimental ./experimental.jl:322
│ [3] showerror(io::IO, ex::MethodError)
│ @ Base ./errorshow.jl:374
│ [4] showerror(io::IOContext{Base.TTY}, ex::MethodError, bt::Vector{Base.StackTraces.StackFrame}; backtrace::Bool)
│ @ Base ./errorshow.jl:106
│ [5] show_exception_stack(io::IOContext{Base.TTY}, stack::Base.ExceptionStack)
│ @ Base ./errorshow.jl:1017
│ [6] display_error(io::IOContext{Base.TTY}, stack::Base.ExceptionStack)
│ @ Base ./client.jl:117
│ [7] repl_display_error(errio::IO, errval::Any)
│ @ REPL ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/REPL/src/REPL.jl:565
│ [8] print_response(errio::IO, response::Any, backend::Union{Nothing, REPL.REPLBackendRef}, show_value::Bool, have_color::Bool, specialdisplay::Union{Nothing, AbstractDisplay})
│ @ REPL ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/REPL/src/REPL.jl:582
│ [9] (::REPL.var"#print_response##0#print_response##1"{REPL.LineEditREPL, Pair{Any, Bool}, Bool, Bool})(io::Any)
│ @ REPL ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/REPL/src/REPL.jl:556
│ [10] with_repl_linfo(f::Any, repl::REPL.LineEditREPL)
│ @ REPL ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/REPL/src/REPL.jl:829
│ [11] print_response(repl::REPL.AbstractREPL, response::Any, show_value::Bool, have_color::Bool)
│ @ REPL ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/REPL/src/REPL.jl:554
│ [12] (::REPL.var"#do_respond#73"{Bool, Bool, REPL.var"#setup_interface#JuliaLang#12#setup_interface#JuliaLang#13"{REPL.LineEditREPL, REPL.REPLHistoryProvider}, REPL.LineEditREPL, REPL.LineEdit.Prompt})(s::REPL.LineEdit.MIState, buf::Any, ok::Bool)
│ @ REPL ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/REPL/src/REPL.jl:1186
│ [13] run_interface(terminal::REPL.Terminals.TextTerminal, m::REPL.LineEdit.ModalInterface, s::REPL.LineEdit.MIState)
│ @ REPL.LineEdit ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/REPL/src/LineEdit.jl:2852
│ [14] run_frontend(repl::REPL.LineEditREPL, backend::REPL.REPLBackendRef)
│ @ REPL ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/REPL/src/REPL.jl:1657
│ [15] (::REPL.var"JuliaLang#61#62"{REPL.LineEditREPL, REPL.REPLBackendRef})()
│ @ REPL ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/REPL/src/REPL.jl:648
└ @ Base.Experimental experimental.jl:325
Stacktrace:
[1] top-level scope
@ REPL[1]:1
```
After:
```julia-repl
julia> missing(1)
ERROR: MethodError: objects of type Missing are not callable.
In case you did not try calling it explicitly, check if a Missing has been passed as an argument to a method that expects a callable instead.
The object of type `Missing` exists, but no method is defined for this combination of argument types when trying to treat it as a callable object.
Stacktrace:
[1] top-level scope
@ REPL[3]:1
```
---------
Co-authored-by: Ian Butterworth <i.r.butterworth@gmail.com>
adienes
pushed a commit
that referenced
this pull request
May 16, 2025
…ze arguments (JuliaLang#58426) When we capture a variable without boxing, we always generate a type parameter for the closure. This probably isn't what the user wants if the captured variable is an argument marked `@nospecialize`. Before: ``` julia> K(@nospecialize(x)) = @nospecialize(y) -> x K (generic function with 1 method) julia> f, g = K(1), K("a") (var"#K##0#K##1"{Int64}(1), var"#K##0#K##1"{String}("a")) julia> f(2), g(2) (1, "a") julia> methods(f)[1].specializations svec(MethodInstance for (::var"#K##0#K##1"{Int64})(::Any), MethodInstance for (::var"#K##0#K##1"{String})(::Any), nothing, nothing, nothing, nothing, nothing) julia> fieldtypes(typeof(f)), fieldtypes(typeof(g)) ((Int64,), (String,)) ``` After: ``` julia> K(@nospecialize(x)) = @nospecialize(y) -> x K (generic function with 1 method) julia> f, g = K(1), K("a") (var"#K##0#K##1"(1), var"#K##0#K##1"("a")) julia> f(2), g(2) (1, "a") julia> methods(f)[1].specializations MethodInstance for (::var"#K##0#K##1")(::Any) julia> fieldtypes(typeof(f)), fieldtypes(typeof(g)) ((Any,), (Any,)) ```
adienes
pushed a commit
that referenced
this pull request
Jul 3, 2025
Use an atomic fetch and add to fix a data race in `Module()` identified by tsan: ``` ./usr/bin/julia -t4,0 --gcthreads=1 -e 'Threads.@threads for i=1:100 Module() end' ================== WARNING: ThreadSanitizer: data race (pid=5575) Write of size 4 at 0xffff9bf9bd28 by thread T9: #0 jl_new_module__ /home/user/c/julia/src/module.c:487:22 (libjulia-internal.so.1.13+0x897d4) #1 jl_new_module_ /home/user/c/julia/src/module.c:527:22 (libjulia-internal.so.1.13+0x897d4) #2 jl_f_new_module /home/user/c/julia/src/module.c:649:22 (libjulia-internal.so.1.13+0x8a968) #3 <null> <null> (0xffff76a21164) #4 <null> <null> (0xffff76a1f074) #5 <null> <null> (0xffff76a1f0c4) #6 _jl_invoke /home/user/c/julia/src/gf.c (libjulia-internal.so.1.13+0x5ea04) #7 ijl_apply_generic /home/user/c/julia/src/gf.c:3892:12 (libjulia-internal.so.1.13+0x5ea04) #8 jl_apply /home/user/c/julia/src/julia.h:2343:12 (libjulia-internal.so.1.13+0x9e4c4) #9 start_task /home/user/c/julia/src/task.c:1249:19 (libjulia-internal.so.1.13+0x9e4c4) Previous write of size 4 at 0xffff9bf9bd28 by thread T10: #0 jl_new_module__ /home/user/c/julia/src/module.c:487:22 (libjulia-internal.so.1.13+0x897d4) #1 jl_new_module_ /home/user/c/julia/src/module.c:527:22 (libjulia-internal.so.1.13+0x897d4) #2 jl_f_new_module /home/user/c/julia/src/module.c:649:22 (libjulia-internal.so.1.13+0x8a968) #3 <null> <null> (0xffff76a21164) #4 <null> <null> (0xffff76a1f074) #5 <null> <null> (0xffff76a1f0c4) #6 _jl_invoke /home/user/c/julia/src/gf.c (libjulia-internal.so.1.13+0x5ea04) #7 ijl_apply_generic /home/user/c/julia/src/gf.c:3892:12 (libjulia-internal.so.1.13+0x5ea04) #8 jl_apply /home/user/c/julia/src/julia.h:2343:12 (libjulia-internal.so.1.13+0x9e4c4) #9 start_task /home/user/c/julia/src/task.c:1249:19 (libjulia-internal.so.1.13+0x9e4c4) Location is global 'jl_new_module__.mcounter' of size 4 at 0xffff9bf9bd28 (libjulia-internal.so.1.13+0x3dbd28) ```
adienes
pushed a commit
that referenced
this pull request
Jul 16, 2025
Simplify `workqueue_for`. While not strictly necessary, the acquire load
in `getindex(once::OncePerThread{T,F}, tid::Integer)` makes
ThreadSanitizer happy. With the existing implementation, we get false
positives whenever a thread other than the one that originally allocated
the array reads it:
```
==================
WARNING: ThreadSanitizer: data race (pid=6819)
Atomic read of size 8 at 0xffff86bec058 by main thread:
#0 getproperty Base_compiler.jl:57 (sys.so+0x113b478)
#1 julia_pushNOT._1925 task.jl:868 (sys.so+0x113b478)
#2 julia_enq_work_1896 task.jl:969 (sys.so+0x5cd218)
#3 schedule task.jl:983 (sys.so+0x892294)
#4 macro expansion threadingconstructs.jl:522 (sys.so+0x892294)
#5 julia_start_profile_listener_60681 Base.jl:355 (sys.so+0x892294)
#6 julia___init___60641 Base.jl:392 (sys.so+0x1178dc)
#7 jfptr___init___60642 <null> (sys.so+0x118134)
#8 _jl_invoke /home/user/c/julia/src/gf.c (libjulia-internal.so.1.13+0x5e9a4)
#9 ijl_apply_generic /home/user/c/julia/src/gf.c:3892:12 (libjulia-internal.so.1.13+0x5e9a4)
#10 jl_apply /home/user/c/julia/src/julia.h:2343:12 (libjulia-internal.so.1.13+0xbba74)
#11 jl_module_run_initializer /home/user/c/julia/src/toplevel.c:68:13 (libjulia-internal.so.1.13+0xbba74)
JuliaLang#12 _finish_jl_init_ /home/user/c/julia/src/init.c:632:13 (libjulia-internal.so.1.13+0x9c0fc)
JuliaLang#13 ijl_init_ /home/user/c/julia/src/init.c:783:5 (libjulia-internal.so.1.13+0x9bcf4)
JuliaLang#14 jl_repl_entrypoint /home/user/c/julia/src/jlapi.c:1125:5 (libjulia-internal.so.1.13+0xf7ec8)
JuliaLang#15 jl_load_repl /home/user/c/julia/cli/loader_lib.c:601:12 (libjulia.so.1.13+0x11934)
JuliaLang#16 main /home/user/c/julia/cli/loader_exe.c:58:15 (julia+0x10dc20)
Previous write of size 8 at 0xffff86bec058 by thread T2:
#0 IntrusiveLinkedListSynchronized task.jl:863 (sys.so+0x78d220)
#1 macro expansion task.jl:932 (sys.so+0x78d220)
#2 macro expansion lock.jl:376 (sys.so+0x78d220)
#3 julia_workqueue_for_1933 task.jl:924 (sys.so+0x78d220)
#4 julia_wait_2048 task.jl:1204 (sys.so+0x6255ac)
#5 julia_task_done_hook_49205 task.jl:839 (sys.so+0x128fdc0)
#6 jfptr_task_done_hook_49206 <null> (sys.so+0x902218)
#7 _jl_invoke /home/user/c/julia/src/gf.c (libjulia-internal.so.1.13+0x5e9a4)
#8 ijl_apply_generic /home/user/c/julia/src/gf.c:3892:12 (libjulia-internal.so.1.13+0x5e9a4)
#9 jl_apply /home/user/c/julia/src/julia.h:2343:12 (libjulia-internal.so.1.13+0x9c79c)
#10 jl_finish_task /home/user/c/julia/src/task.c:345:13 (libjulia-internal.so.1.13+0x9c79c)
#11 jl_threadfun /home/user/c/julia/src/scheduler.c:122:5 (libjulia-internal.so.1.13+0xe7db8)
Thread T2 (tid=6824, running) created by main thread at:
#0 pthread_create <null> (julia+0x85f88)
#1 uv_thread_create_ex /workspace/srcdir/libuv/src/unix/thread.c:172 (libjulia-internal.so.1.13+0x1a8d70)
#2 _finish_jl_init_ /home/user/c/julia/src/init.c:618:5 (libjulia-internal.so.1.13+0x9c010)
#3 ijl_init_ /home/user/c/julia/src/init.c:783:5 (libjulia-internal.so.1.13+0x9bcf4)
#4 jl_repl_entrypoint /home/user/c/julia/src/jlapi.c:1125:5 (libjulia-internal.so.1.13+0xf7ec8)
#5 jl_load_repl /home/user/c/julia/cli/loader_lib.c:601:12 (libjulia.so.1.13+0x11934)
#6 main /home/user/c/julia/cli/loader_exe.c:58:15 (julia+0x10dc20)
SUMMARY: ThreadSanitizer: data race Base_compiler.jl:57 in getproperty
==================
```
adienes
pushed a commit
that referenced
this pull request
Sep 26, 2025
Fixes ``` patching file src/ptrace/_UPT_ptrauth_insn_mask.c Hunk #1 FAILED at 49. 1 out of 1 hunk FAILED -- saving rejects to file src/ptrace/_UPT_ptrauth_insn_mask.c.rej make[1]: *** [[buildroot]/deps/unwind.mk:43: [buildroot]/deps/srccache/libunwind-1.8.3/libunwind-missing-parameter-names.patch-applied] Error 1 make[1]: *** Waiting for unfinished jobs.... ``` because the patch was backported to the v1.8 branch.
adienes
pushed a commit
that referenced
this pull request
Oct 10, 2025
Use an atomic fetch and add to fix a data race in `Module()` identified by tsan: ``` ./usr/bin/julia -t4,0 --gcthreads=1 -e 'Threads.@threads for i=1:100 Module() end' ================== WARNING: ThreadSanitizer: data race (pid=5575) Write of size 4 at 0xffff9bf9bd28 by thread T9: #0 jl_new_module__ /home/user/c/julia/src/module.c:487:22 (libjulia-internal.so.1.13+0x897d4) #1 jl_new_module_ /home/user/c/julia/src/module.c:527:22 (libjulia-internal.so.1.13+0x897d4) #2 jl_f_new_module /home/user/c/julia/src/module.c:649:22 (libjulia-internal.so.1.13+0x8a968) #3 <null> <null> (0xffff76a21164) #4 <null> <null> (0xffff76a1f074) #5 <null> <null> (0xffff76a1f0c4) #6 _jl_invoke /home/user/c/julia/src/gf.c (libjulia-internal.so.1.13+0x5ea04) #7 ijl_apply_generic /home/user/c/julia/src/gf.c:3892:12 (libjulia-internal.so.1.13+0x5ea04) #8 jl_apply /home/user/c/julia/src/julia.h:2343:12 (libjulia-internal.so.1.13+0x9e4c4) #9 start_task /home/user/c/julia/src/task.c:1249:19 (libjulia-internal.so.1.13+0x9e4c4) Previous write of size 4 at 0xffff9bf9bd28 by thread T10: #0 jl_new_module__ /home/user/c/julia/src/module.c:487:22 (libjulia-internal.so.1.13+0x897d4) #1 jl_new_module_ /home/user/c/julia/src/module.c:527:22 (libjulia-internal.so.1.13+0x897d4) #2 jl_f_new_module /home/user/c/julia/src/module.c:649:22 (libjulia-internal.so.1.13+0x8a968) #3 <null> <null> (0xffff76a21164) #4 <null> <null> (0xffff76a1f074) #5 <null> <null> (0xffff76a1f0c4) #6 _jl_invoke /home/user/c/julia/src/gf.c (libjulia-internal.so.1.13+0x5ea04) #7 ijl_apply_generic /home/user/c/julia/src/gf.c:3892:12 (libjulia-internal.so.1.13+0x5ea04) #8 jl_apply /home/user/c/julia/src/julia.h:2343:12 (libjulia-internal.so.1.13+0x9e4c4) #9 start_task /home/user/c/julia/src/task.c:1249:19 (libjulia-internal.so.1.13+0x9e4c4) Location is global 'jl_new_module__.mcounter' of size 4 at 0xffff9bf9bd28 (libjulia-internal.so.1.13+0x3dbd28) ``` (cherry picked from commit 9039555)
adienes
pushed a commit
that referenced
this pull request
Dec 16, 2025
…liaLang#60259) If a method we precompile instantiates a closure struct, and there is a single compileable signature for the closure method, we can compile the closure method with few downsides. For example: ```julia f(x::T) where {T <: Number} = y::T -> x+y precompile(f, (Int,)) f(1)(2) ``` Before, the closure method would not get precompiled, resulting in `--trace-compile` output: `precompile(Tuple{Foo.var"#f##0#f##1"{Int64, Int64}, Int64})` This idea was motivated by trying to make the REPL precompile script more "static". Creating a task with `@task` or `@async` expands into a no-argument lambda, which won't get precompiled with the containing method.
adienes
pushed a commit
that referenced
this pull request
Dec 30, 2025
As a bonus, also change `T == BigInt` to `T === BigInt`.
Fixes this:
```
│┌ load_artifacts_toml(artifacts_toml::String) @ Artifacts /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-R17H3W25T9.0/build/default-honeycrisp-R17H3W25T9-0/julialang/julia-release-1-dot-12/usr/share/julia/stdlib/v1.12/Artifacts/src/Artifacts.jl:315
││┌ load_artifacts_toml(artifacts_toml::String; pkg_uuid::Nothing) @ Artifacts /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-R17H3W25T9.0/build/default-honeycrisp-R17H3W25T9-0/julialang/julia-release-1-dot-12/usr/share/julia/stdlib/v1.12/Artifacts/src/Artifacts.jl:317
│││┌ parse_toml(path::String) @ Artifacts /Users/julia/.julia/scratchspaces/a66863c6-20e8-4ff4-8a62-49f30b1f605e/agent-cache/default-honeycrisp-R17H3W25T9.0/build/default-honeycrisp-R17H3W25T9-0/julialang/julia-release-1-dot-12/usr/share/julia/stdlib/v1.12/Artifacts/src/Artifacts.jl:26
││││┌ parsed_toml(project_file::String) @ Base ./loading.jl:276
│││││┌ parsed_toml(project_file::String, toml_cache::Base.TOMLCache{nothing}, toml_lock::ReentrantLock) @ Base ./loading.jl:278
││││││┌ lock(f::Base.var"#parsed_toml##0#parsed_toml##1"{String, Base.TOMLCache{nothing}}, l::ReentrantLock) @ Base ./lock.jl:335
│││││││┌ (::Base.var"#parsed_toml##0#parsed_toml##1"{String, Base.TOMLCache{nothing}})() @ Base ./loading.jl:281
││││││││┌ Base.CachedTOMLDict(p::Base.TOML.Parser{nothing}, path::String) @ Base ./loading.jl:222
│││││││││┌ parse(l::Base.TOML.Parser{nothing}) @ Base.TOML ./toml_parser.jl:444
││││││││││┌ tryparse(l::Base.TOML.Parser{nothing}) @ Base.TOML ./toml_parser.jl:453
│││││││││││┌ parse_toplevel(l::Base.TOML.Parser{nothing}) @ Base.TOML ./toml_parser.jl:157
││││││││││││┌ parse_array(l::Base.TOML.Parser{nothing}) @ Base.TOML ./toml_parser.jl:719
│││││││││││││┌ getproperty(x::Nothing, f::Symbol) @ Base ./Base_compiler.jl:54
││││││││││││││ invalid builtin function call: getfield(x::Nothing, f::Symbol)
│││││││││││││└────────────────────
```
adienes
pushed a commit
that referenced
this pull request
Feb 25, 2026
# Overview This PR overhauls the way linking works in Julia, both in the JIT and AOT. The point is to enable us to generate LLVM IR that depends only on the source IR, eliminating both nondeterminism and statefulness. This serves two purposes. First, if the IR is predictable, we can cache compile objects using the bitcode hash as a key, like how the ThinLTO cache works. JuliaLang#58592 was an early experiment along these lines. Second, we can reuse work that was done in a previous session, like pkgimages, but for the JIT. We accomplish this by generating names that are unique only within the current LLVM module, removing most uses of the `globalUniqueGeneratedNames` counter. The replacement for `jl_codegen_params_t`, `jl_codegen_output_t`, represents a Julia "translation unit", and tracks the information we'll need to link the compiled module into the running session. When linking, we manipulate the JITLink [LinkGraph](https://llvm.org/docs/JITLink.html#linkgraph) (after compilation) instead of renaming functions in the LLVM IR (before). ## Example ``` julia> @noinline foo(x) = x + 2.0 baz(x) = foo(foo(x)) code_llvm(baz, (Int64,); dump_module=true, optimize=false) ``` Nightly: ```llvm [...] @"+Core.Float64#774" = private unnamed_addr constant ptr @"+Core.Float64#774.jit" @"+Core.Float64#774.jit" = private alias ptr, inttoptr (i64 4797624416 to ptr) ; Function Signature: baz(Int64) ; @ REPL[1]:2 within `baz` define double @julia_baz_772(i64 signext %"x::Int64") #0 { top: %pgcstack = call ptr @julia.get_pgcstack() %0 = call double @j_foo_775(i64 signext %"x::Int64") %1 = call double @j_foo_776(double %0) ret double %1 } ; Function Attrs: noinline optnone define nonnull ptr @jfptr_baz_773(ptr %"function::Core.Function", ptr noalias nocapture noundef readonly %"args::Any[]", i32 %"nargs::UInt32") #1 { top: %pgcstack = call ptr @julia.get_pgcstack() %0 = getelementptr inbounds i8, ptr %"args::Any[]", i32 0 %1 = load ptr, ptr %0, align 8 %.unbox = load i64, ptr %1, align 8 %2 = call double @julia_baz_772(i64 signext %.unbox) %"+Core.Float64#774" = load ptr, ptr @"+Core.Float64#774", align 8 %Float64 = ptrtoint ptr %"+Core.Float64#774" to i64 %3 = inttoptr i64 %Float64 to ptr %current_task = getelementptr inbounds i8, ptr %pgcstack, i32 -152 %"box::Float64" = call noalias nonnull align 8 dereferenceable(8) ptr @julia.gc_alloc_obj(ptr %current_task, i64 8, ptr %3) #5 store double %2, ptr %"box::Float64", align 8 ret ptr %"box::Float64" } [...] ``` Diff after this PR. Notice how each symbol gets the lowest possible integer suffix that will make it unique to the module, and how the two specializations for `foo` get different names: ```diff @@ -4,18 +4,18 @@ target triple = "arm64-apple-darwin24.6.0" -@"+Core.Float64#774" = external global ptr +@"+Core.Float64#_0" = external global ptr ; Function Signature: baz(Int64) ; @ REPL[1]:2 within `baz` -define double @julia_baz_772(i64 signext %"x::Int64") #0 { +define double @julia_baz_0(i64 signext %"x::Int64") #0 { top: %pgcstack = call ptr @julia.get_pgcstack() - %0 = call double @j_foo_775(i64 signext %"x::Int64") - %1 = call double @j_foo_776(double %0) + %0 = call double @j_foo_0(i64 signext %"x::Int64") + %1 = call double @j_foo_1(double %0) ret double %1 } ; Function Attrs: noinline optnone -define nonnull ptr @jfptr_baz_773(ptr %"function::Core.Function", ptr noalias nocapture noundef readonly %"args::Any[]", i32 %"nargs::UInt32") #1 { +define nonnull ptr @jfptr_baz_0(ptr %"function::Core.Function", ptr noalias nocapture noundef readonly %"args::Any[]", i32 %"nargs::UInt32") #1 { top: %pgcstack = call ptr @julia.get_pgcstack() @@ -23,7 +23,7 @@ %1 = load ptr, ptr %0, align 8 %.unbox = load i64, ptr %1, align 8 - %2 = call double @julia_baz_772(i64 signext %.unbox) - %"+Core.Float64#774" = load ptr, ptr @"+Core.Float64#774", align 8 - %Float64 = ptrtoint ptr %"+Core.Float64#774" to i64 + %2 = call double @julia_baz_0(i64 signext %.unbox) + %"+Core.Float64#_0" = load ptr, ptr @"+Core.Float64#_0", align 8 + %Float64 = ptrtoint ptr %"+Core.Float64#_0" to i64 %3 = inttoptr i64 %Float64 to ptr %current_task = getelementptr inbounds i8, ptr %pgcstack, i32 -152 @@ -39,8 +39,8 @@ ; Function Signature: foo(Int64) -declare double @j_foo_775(i64 signext) #3 +declare double @j_foo_0(i64 signext) #3 ; Function Signature: foo(Float64) -declare double @j_foo_776(double) #4 +declare double @j_foo_1(double) #4 attributes #0 = { "frame-pointer"="all" "julia.fsig"="baz(Int64)" "probe-stack"="inline-asm" } ``` ## List of changes - Many sources of statefulness and nondeterminism in the emitted LLVM IR have been eliminated, namely: - Function symbols defined for CodeInstances - Global symbols referring to data on the Julia heap - Undefined function symbols referring to invoked external CodeInstances - `jl_codeinst_params_t` has become `jl_codegen_output_t`. It now represents one Julia "translation unit". More than one CodeInstance can be emitted to the same `jl_codegen_output_t`, if desired, though in the JIT every CI gets its own right now. One motivation behind this is to allow us to emit code on multiple threads and avoid the bitcode serialize/deserialize step we currently do, if that proves worthwhile. When we are done emitting to a `jl_codegen_output_t`, we call `.finish()`, which discards the intermediate state and returns only the LLVM module and the info needed for linking (`jl_linker_info_t`). - The new `JLMaterializationUnit` wraps emitting Julia LLVM modules and the associated `jl_linker_info_t`. It informs ORC that we can materialize symbols for the CIs defined by that output, and picks globally unique names for them. When it is materialized, it resolves all the call targets and generates trampolines for CodeInstances that are invoked but have the wrong calling convention, or are not yet compiled. - We now postpone linking decisions to after codegen whenever possible. For example, `emit_invoke` no longer tries to find a compiled version of the CodeInstance, and it no longer generates trampolines to adapt calling conventions. `jl_analyze_workqueue`'s job has been absorbed into `JuliaOJIT::linkOutput`. - Some `image_codegen` differences have been removed: - Codegen no longer cares if a compiled CodeInstance came from an image. During ahead-of-time linking, we generate thunk functions that load the address from the fvars table. - In `jl_emit_native_impl`, emit every CodeInstance into one `jl_codegen_output_t`. We now defer the creation of the `llvm::Linker` for llvmcalls, which has construction cost that grows with the size of the destination module, until the very end. - RTDyld is removed completely, since we cannot control linking like we can with JITLink. Since JuliaLang#60105, platforms that previous used the optimized memory manager now use the new one. ### General refactoring - Adapt the `jl_callingconv_t` enum from `staticdata.c` into `jl_invoke_api_t` and use it in more places. There is one enumerator for each special `jl_callptr_t` function that can go in a CodeInstance's `invoke` field, as well as one that indicates an invoke wrapper should be there. There is a convenience function for reading an invoke pointer and getting the API type, and vice versa. - Avoid using magic string values, and try to directly pass pointers to LLVM `Function *` or ORC string pool entries when possible. ## Future work - `DLSymOptimizer` should be mostly removed, in favour of emitting raw ccalls and redirecting them to the appropriate target during linking. - We should support ahead-of-time linking multiple `jl_codegen_output_t`s together, in order to parallelize LLVM IR emission when compiling a system image. - We still pass strings to `emit_call_specfun_other`, even though the prototype for the function is now created by `jl_codegen_output_t::get_call_target`. We should hold on to the calling convention info so it doesn't have to be recomputed.
adienes
pushed a commit
that referenced
this pull request
Mar 27, 2026
…#61330) Should prevent these invalidations seen when loading CurveFit.jl on 1.13: ```julia inserting merge!(m::DataStructures.SortedDict{K, D, Ord}, others::AbstractDict{K, D}...) where {K, D, Ord<:Ordering} @ DataStructures ~/.julia/packages/DataStructures/qUuAY/src/sorted_dict.jl:473 invalidated: mt_backedges: 1: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator{_A, Base.Compiler.IRShow.var"JuliaLang#30#31"} where _A, ::Any) where {K, V} (0 children) 2: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator, ::Any) where {K, V} (0 children) 3: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator, ::Any) where {K, V} (0 children) 4: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator{I, Base.var"#Generator##0#Generator##1"{NonlinearSolveBase.Utils.var"#norm_op##2#norm_op##3"{typeof(+)}}} where I<:(Base.Iterators.Zip{Is} where Is<:Tuple{Base.AbstractBroadcasted, Base.AbstractBroadcasted}), ::Tuple{Tuple{Any, Any}, Tuple{Any, Any}}) where {K, V} (0 children) 5: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator{_A, F} where {_A, F<:DifferentiationInterface.var"#_prepare_jacobian_aux##0#_prepare_jacobian_aux##1"}, ::Any) where {K, V} (0 children) 6: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator{_A, F} where {_A, F<:DifferentiationInterface.var"#_prepare_jacobian_aux#JuliaLang#12#_prepare_jacobian_aux#JuliaLang#13"}, ::Any) where {K, V} (0 children) 7: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator{I, Base.var"#Generator##0#Generator##1"{NonlinearSolveBase.Utils.var"#norm_op##2#norm_op##3"{typeof(+)}}} where I<:(Base.Iterators.Zip{Is} where Is<:Tuple{Base.AbstractBroadcasted, Vector{Float64}}), ::Tuple{Tuple{Any, Any}, Int64}) where {K, V} (0 children) backedges: 1: superseding merge!(d::AbstractDict, others::AbstractDict...) @ Base abstractdict.jl:224 with MethodInstance for merge!(::AbstractDict, ::AbstractDict) (1126 children) inserting merge(d::OrderedCollections.OrderedDict, others::AbstractDict...) @ OrderedCollections ~/.julia/dev/OrderedCollections/src/ordered_dict.jl:488 invalidated: mt_backedges: <38 invalidations I deleted because they have enormous types> backedges: 1: superseding merge(d::AbstractDict, others::AbstractDict...) @ Base abstractdict.jl:359 with MethodInstance for merge(::AbstractDict, ::Base.Pairs{Symbol, Union{}, Nothing, @NamedTuple{}}) (21 children) 2: superseding merge(d::AbstractDict, others::AbstractDict...) @ Base abstractdict.jl:359 with MethodInstance for merge(::AbstractDict, ::Base.Pairs{Symbol, _A, Nothing, A} where {_A, A<:NamedTuple}) (2489 children) ``` Partial alternative to JuliaLang#61329. I added `mergewith()` and `mergewith!()` to be on the safe side.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.