Skip to content

Update diagonal.jl#12

Merged
N5N3 merged 1 commit intoN5N3-patch-1from
N5N3-patch-2
Sep 23, 2021
Merged

Update diagonal.jl#12
N5N3 merged 1 commit intoN5N3-patch-1from
N5N3-patch-2

Conversation

@N5N3
Copy link
Copy Markdown
Owner

@N5N3 N5N3 commented Sep 23, 2021

No description provided.

@N5N3 N5N3 merged commit 6d17e3e into N5N3-patch-1 Sep 23, 2021
@N5N3 N5N3 deleted the N5N3-patch-2 branch September 25, 2021 06:34
N5N3 pushed a commit that referenced this pull request Mar 28, 2022
Follows up JuliaLang#44708 -- in that PR I missed the most obvious optimization
opportunity, i.e. we can safely eliminate `isdefined` checks when all
fields are defined at allocation site.
This change allows us to eliminate capturing closure constructions when
the body and callsite of capture closure is available within a optimized
frame, e.g.:
```julia
function abmult(r::Int, x0)
    if r < 0
        r = -r
    end
    f = x -> x * r
    return @inline f(x0)
end
```
```diff
diff --git a/_master.jl b/_pr.jl
index ea06d865b75..c38f221090f 100644
--- a/_master.jl
+++ b/_pr.jl
@@ -1,24 +1,19 @@
 julia> @code_typed abmult(-3, 3)
 CodeInfo(
-1 ── %1  = Core.Box::Type{Core.Box}
-│    %2  = %new(%1, r@_2)::Core.Box
-│    %3  = Core.isdefined(%2, :contents)::Bool
-└───       goto #3 if not %3
+1 ──       goto #3 if not true
 2 ──       goto #4
 3 ──       $(Expr(:throw_undef_if_not, :r, false))::Any
-4 ┄─ %7  = (r@_2 < 0)::Any
-└───       goto #9 if not %7
-5 ── %9  = Core.isdefined(%2, :contents)::Bool
-└───       goto #7 if not %9
+4 ┄─ %4  = (r@_2 < 0)::Any
+└───       goto #9 if not %4
+5 ──       goto #7 if not true
 6 ──       goto #8
 7 ──       $(Expr(:throw_undef_if_not, :r, false))::Any
-8 ┄─ %13 = -r@_2::Any
-9 ┄─ %14 = φ (#4 => r@_2, #8 => %13)::Any
-│    %15 = Core.isdefined(%2, :contents)::Bool
-└───       goto #11 if not %15
+8 ┄─ %9  = -r@_2::Any
+9 ┄─ %10 = φ (#4 => r@_2, #8 => %9)::Any
+└───       goto #11 if not true
 10 ─       goto #12
 11 ─       $(Expr(:throw_undef_if_not, :r, false))::Any
-12 ┄ %19 = (x0 * %14)::Any
+12 ┄ %14 = (x0 * %10)::Any
 └───       goto #13
-13 ─       return %19
+13 ─       return %14
 ) => Any
```
N5N3 pushed a commit that referenced this pull request Jun 29, 2022
When calling `jl_error()` or `jl_errorf()`, we must check to see if we
are so early in the bringup process that it is dangerous to attempt to
construct a backtrace because the data structures used to provide line
information are not properly setup.

This can be easily triggered by running:

```
julia -C invalid
```

On an `i686-linux-gnu` build, this will hit the "Invalid CPU Name"
branch in `jitlayers.cpp`, which calls `jl_errorf()`.  This in turn
calls `jl_throw()`, which will eventually call `jl_DI_for_fptr` as part
of the backtrace printing process, which fails as the object maps are
not fully initialized.  See the below `gdb` stacktrace for details:

```
$ gdb -batch -ex 'r' -ex 'bt' --args ./julia -C invalid
...
fatal: error thrown and no exception handler available.
ErrorException("Invalid CPU name "invalid".")

Thread 1 "julia" received signal SIGSEGV, Segmentation fault.
0xf75bd665 in std::_Rb_tree<unsigned int, std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo>, std::_Select1st<std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo> >, std::greater<unsigned int>, std::allocator<std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo> > >::lower_bound (__k=<optimized out>, this=0x248) at /usr/local/i686-linux-gnu/include/c++/9.1.0/bits/stl_tree.h:1277
1277    /usr/local/i686-linux-gnu/include/c++/9.1.0/bits/stl_tree.h: No such file or directory.
 #0  0xf75bd665 in std::_Rb_tree<unsigned int, std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo>, std::_Select1st<std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo> >, std::greater<unsigned int>, std::allocator<std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo> > >::lower_bound (__k=<optimized out>, this=0x248) at /usr/local/i686-linux-gnu/include/c++/9.1.0/bits/stl_tree.h:1277
 #1  std::map<unsigned int, JITDebugInfoRegistry::ObjectInfo, std::greater<unsigned int>, std::allocator<std::pair<unsigned int const, JITDebugInfoRegistry::ObjectInfo> > >::lower_bound (__x=<optimized out>, this=0x248) at /usr/local/i686-linux-gnu/include/c++/9.1.0/bits/stl_map.h:1258
 #2  jl_DI_for_fptr (fptr=4155049385, symsize=symsize@entry=0xffffcfa8, slide=slide@entry=0xffffcfa0, Section=Section@entry=0xffffcfb8, context=context@entry=0xffffcf94) at /cache/build/default-amdci5-4/julialang/julia-master/src/debuginfo.cpp:1181
 #3  0xf75c056a in jl_getFunctionInfo_impl (frames_out=0xffffd03c, pointer=4155049385, skipC=0, noInline=0) at /cache/build/default-amdci5-4/julialang/julia-master/src/debuginfo.cpp:1210
 #4  0xf7a6ca98 in jl_print_native_codeloc (ip=4155049385) at /cache/build/default-amdci5-4/julialang/julia-master/src/stackwalk.c:636
 #5  0xf7a6cd54 in jl_print_bt_entry_codeloc (bt_entry=0xf0798018) at /cache/build/default-amdci5-4/julialang/julia-master/src/stackwalk.c:657
 #6  jlbacktrace () at /cache/build/default-amdci5-4/julialang/julia-master/src/stackwalk.c:1090
 #7  0xf7a3cd2b in ijl_no_exc_handler (e=0xf0794010) at /cache/build/default-amdci5-4/julialang/julia-master/src/task.c:605
 #8  0xf7a3d10a in throw_internal (ct=ct@entry=0xf070c010, exception=<optimized out>, exception@entry=0xf0794010) at /cache/build/default-amdci5-4/julialang/julia-master/src/task.c:638
 #9  0xf7a3d330 in ijl_throw (e=0xf0794010) at /cache/build/default-amdci5-4/julialang/julia-master/src/task.c:654
 #10 0xf7a905aa in ijl_errorf (fmt=fmt@entry=0xf7647cd4 "Invalid CPU name \"%s\".") at /cache/build/default-amdci5-4/julialang/julia-master/src/rtutils.c:77
 #11 0xf75a4b22 in (anonymous namespace)::createTargetMachine () at /cache/build/default-amdci5-4/julialang/julia-master/src/jitlayers.cpp:823
 #12 JuliaOJIT::JuliaOJIT (this=<optimized out>) at /cache/build/default-amdci5-4/julialang/julia-master/src/jitlayers.cpp:1044
 #13 0xf7531793 in jl_init_llvm () at /cache/build/default-amdci5-4/julialang/julia-master/src/codegen.cpp:8585
 #14 0xf75318a8 in jl_init_codegen_impl () at /cache/build/default-amdci5-4/julialang/julia-master/src/codegen.cpp:8648
 #15 0xf7a51a52 in jl_restore_system_image_from_stream (f=<optimized out>) at /cache/build/default-amdci5-4/julialang/julia-master/src/staticdata.c:2131
 #16 0xf7a55c03 in ijl_restore_system_image_data (buf=0xe859c1c0 <jl_system_image_data> "8'\031\003", len=125161105) at /cache/build/default-amdci5-4/julialang/julia-master/src/staticdata.c:2184
 #17 0xf7a55cf9 in jl_load_sysimg_so () at /cache/build/default-amdci5-4/julialang/julia-master/src/staticdata.c:424
 #18 ijl_restore_system_image (fname=0x80a0900 "/build/bk_download/julia-d78fdad601/lib/julia/sys.so") at /cache/build/default-amdci5-4/julialang/julia-master/src/staticdata.c:2157
 #19 0xf7a3bdfc in _finish_julia_init (rel=rel@entry=JL_IMAGE_JULIA_HOME, ct=<optimized out>, ptls=<optimized out>) at /cache/build/default-amdci5-4/julialang/julia-master/src/init.c:741
 #20 0xf7a3c8ac in julia_init (rel=<optimized out>) at /cache/build/default-amdci5-4/julialang/julia-master/src/init.c:728
 #21 0xf7a7f61d in jl_repl_entrypoint (argc=<optimized out>, argv=0xffffddf4) at /cache/build/default-amdci5-4/julialang/julia-master/src/jlapi.c:705
 #22 0x080490a7 in main (argc=3, argv=0xffffddf4) at /cache/build/default-amdci5-4/julialang/julia-master/cli/loader_exe.c:59
```

To prevent this, we simply avoid calling `jl_errorf` this early in the
process, punting the problem to a later PR that can update guard
conditions within `jl_error*`.
mbauman added a commit that referenced this pull request May 12, 2025
Before:

```julia-repl
julia> missing(1)
ERROR: MethodError: objects of type Missing are not callable.
In case you did not try calling it explicitly, check if a Missing has been passed as an argument to a method that expects a callable instead.
The object of type `Missing` exists, but no method is defined for this combination of argument types when trying to treat it as a callable object.┌ Error: Hint-handler nonsetable_type_hint_handler for MethodError in Base caused an error
│   exception =
│    1-element ExceptionStack:
│    TypeError: non-boolean (Missing) used in boolean context
│    Stacktrace:
│      [1] nonsetable_type_hint_handler(io::Any, ex::Any, arg_types::Any, kwargs::Any)
│        @ Base ./errorshow.jl:1054
│      [2] show_error_hints(::IOContext{Base.TTY}, ::MethodError, ::Vector{Any}, ::Vararg{Vector{Any}})
│        @ Base.Experimental ./experimental.jl:322
│      [3] showerror(io::IO, ex::MethodError)
│        @ Base ./errorshow.jl:374
│      [4] showerror(io::IOContext{Base.TTY}, ex::MethodError, bt::Vector{Base.StackTraces.StackFrame}; backtrace::Bool)
│        @ Base ./errorshow.jl:106
│      [5] show_exception_stack(io::IOContext{Base.TTY}, stack::Base.ExceptionStack)
│        @ Base ./errorshow.jl:1017
│      [6] display_error(io::IOContext{Base.TTY}, stack::Base.ExceptionStack)
│        @ Base ./client.jl:117
│      [7] repl_display_error(errio::IO, errval::Any)
│        @ REPL ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/REPL/src/REPL.jl:565
│      [8] print_response(errio::IO, response::Any, backend::Union{Nothing, REPL.REPLBackendRef}, show_value::Bool, have_color::Bool, specialdisplay::Union{Nothing, AbstractDisplay})
│        @ REPL ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/REPL/src/REPL.jl:582
│      [9] (::REPL.var"#print_response##0#print_response##1"{REPL.LineEditREPL, Pair{Any, Bool}, Bool, Bool})(io::Any)
│        @ REPL ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/REPL/src/REPL.jl:556
│     [10] with_repl_linfo(f::Any, repl::REPL.LineEditREPL)
│        @ REPL ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/REPL/src/REPL.jl:829
│     [11] print_response(repl::REPL.AbstractREPL, response::Any, show_value::Bool, have_color::Bool)
│        @ REPL ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/REPL/src/REPL.jl:554
│     [12] (::REPL.var"#do_respond#73"{Bool, Bool, REPL.var"#setup_interface##12#setup_interface##13"{REPL.LineEditREPL, REPL.REPLHistoryProvider}, REPL.LineEditREPL, REPL.LineEdit.Prompt})(s::REPL.LineEdit.MIState, buf::Any, ok::Bool)
│        @ REPL ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/REPL/src/REPL.jl:1186
│     [13] run_interface(terminal::REPL.Terminals.TextTerminal, m::REPL.LineEdit.ModalInterface, s::REPL.LineEdit.MIState)
│        @ REPL.LineEdit ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/REPL/src/LineEdit.jl:2852
│     [14] run_frontend(repl::REPL.LineEditREPL, backend::REPL.REPLBackendRef)
│        @ REPL ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/REPL/src/REPL.jl:1657
│     [15] (::REPL.var"JuliaLang#61#62"{REPL.LineEditREPL, REPL.REPLBackendRef})()
│        @ REPL ~/.julia/juliaup/julia-nightly/share/julia/stdlib/v1.13/REPL/src/REPL.jl:648
└ @ Base.Experimental experimental.jl:325

Stacktrace:
 [1] top-level scope
   @ REPL[1]:1
```

After:

```julia-repl
julia> missing(1)
ERROR: MethodError: objects of type Missing are not callable.
In case you did not try calling it explicitly, check if a Missing has been passed as an argument to a method that expects a callable instead.
The object of type `Missing` exists, but no method is defined for this combination of argument types when trying to treat it as a callable object.
Stacktrace:
 [1] top-level scope
   @ REPL[3]:1
```

---------

Co-authored-by: Ian Butterworth <i.r.butterworth@gmail.com>
adienes pushed a commit that referenced this pull request Sep 9, 2025
Simplify `workqueue_for`. While not strictly necessary, the acquire load
in `getindex(once::OncePerThread{T,F}, tid::Integer)` makes
ThreadSanitizer happy. With the existing implementation, we get false
positives whenever a thread other than the one that originally allocated
the array reads it:

```
==================
WARNING: ThreadSanitizer: data race (pid=6819)
  Atomic read of size 8 at 0xffff86bec058 by main thread:
    #0 getproperty Base_compiler.jl:57 (sys.so+0x113b478)
    #1 julia_pushNOT._1925 task.jl:868 (sys.so+0x113b478)
    #2 julia_enq_work_1896 task.jl:969 (sys.so+0x5cd218)
    #3 schedule task.jl:983 (sys.so+0x892294)
    #4 macro expansion threadingconstructs.jl:522 (sys.so+0x892294)
    #5 julia_start_profile_listener_60681 Base.jl:355 (sys.so+0x892294)
    #6 julia___init___60641 Base.jl:392 (sys.so+0x1178dc)
    #7 jfptr___init___60642 <null> (sys.so+0x118134)
    #8 _jl_invoke /home/user/c/julia/src/gf.c (libjulia-internal.so.1.13+0x5e9a4)
    #9 ijl_apply_generic /home/user/c/julia/src/gf.c:3892:12 (libjulia-internal.so.1.13+0x5e9a4)
    #10 jl_apply /home/user/c/julia/src/julia.h:2343:12 (libjulia-internal.so.1.13+0xbba74)
    #11 jl_module_run_initializer /home/user/c/julia/src/toplevel.c:68:13 (libjulia-internal.so.1.13+0xbba74)
    #12 _finish_jl_init_ /home/user/c/julia/src/init.c:632:13 (libjulia-internal.so.1.13+0x9c0fc)
    #13 ijl_init_ /home/user/c/julia/src/init.c:783:5 (libjulia-internal.so.1.13+0x9bcf4)
    #14 jl_repl_entrypoint /home/user/c/julia/src/jlapi.c:1125:5 (libjulia-internal.so.1.13+0xf7ec8)
    #15 jl_load_repl /home/user/c/julia/cli/loader_lib.c:601:12 (libjulia.so.1.13+0x11934)
    #16 main /home/user/c/julia/cli/loader_exe.c:58:15 (julia+0x10dc20)

  Previous write of size 8 at 0xffff86bec058 by thread T2:
    #0 IntrusiveLinkedListSynchronized task.jl:863 (sys.so+0x78d220)
    #1 macro expansion task.jl:932 (sys.so+0x78d220)
    #2 macro expansion lock.jl:376 (sys.so+0x78d220)
    #3 julia_workqueue_for_1933 task.jl:924 (sys.so+0x78d220)
    #4 julia_wait_2048 task.jl:1204 (sys.so+0x6255ac)
    #5 julia_task_done_hook_49205 task.jl:839 (sys.so+0x128fdc0)
    #6 jfptr_task_done_hook_49206 <null> (sys.so+0x902218)
    #7 _jl_invoke /home/user/c/julia/src/gf.c (libjulia-internal.so.1.13+0x5e9a4)
    #8 ijl_apply_generic /home/user/c/julia/src/gf.c:3892:12 (libjulia-internal.so.1.13+0x5e9a4)
    #9 jl_apply /home/user/c/julia/src/julia.h:2343:12 (libjulia-internal.so.1.13+0x9c79c)
    #10 jl_finish_task /home/user/c/julia/src/task.c:345:13 (libjulia-internal.so.1.13+0x9c79c)
    #11 jl_threadfun /home/user/c/julia/src/scheduler.c:122:5 (libjulia-internal.so.1.13+0xe7db8)

  Thread T2 (tid=6824, running) created by main thread at:
    #0 pthread_create <null> (julia+0x85f88)
    #1 uv_thread_create_ex /workspace/srcdir/libuv/src/unix/thread.c:172 (libjulia-internal.so.1.13+0x1a8d70)
    #2 _finish_jl_init_ /home/user/c/julia/src/init.c:618:5 (libjulia-internal.so.1.13+0x9c010)
    #3 ijl_init_ /home/user/c/julia/src/init.c:783:5 (libjulia-internal.so.1.13+0x9bcf4)
    #4 jl_repl_entrypoint /home/user/c/julia/src/jlapi.c:1125:5 (libjulia-internal.so.1.13+0xf7ec8)
    #5 jl_load_repl /home/user/c/julia/cli/loader_lib.c:601:12 (libjulia.so.1.13+0x11934)
    #6 main /home/user/c/julia/cli/loader_exe.c:58:15 (julia+0x10dc20)

SUMMARY: ThreadSanitizer: data race Base_compiler.jl:57 in getproperty
==================
```
N5N3 pushed a commit that referenced this pull request Apr 5, 2026
…#61330)

Should prevent these invalidations seen when loading CurveFit.jl on
1.13:
```julia
inserting merge!(m::DataStructures.SortedDict{K, D, Ord}, others::AbstractDict{K, D}...) where {K, D, Ord<:Ordering} @ DataStructures ~/.julia/packages/DataStructures/qUuAY/src/sorted_dict.jl:473 invalidated:
   mt_backedges: 1: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator{_A, Base.Compiler.IRShow.var"JuliaLang#30#31"} where _A, ::Any) where {K, V} (0 children)
                 2: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator, ::Any) where {K, V} (0 children)
                 3: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator, ::Any) where {K, V} (0 children)
                 4: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator{I, Base.var"#Generator##0#Generator##1"{NonlinearSolveBase.Utils.var"#norm_op##2#norm_op##3"{typeof(+)}}} where I<:(Base.Iterators.Zip{Is} where Is<:Tuple{Base.AbstractBroadcasted, Base.AbstractBroadcasted}), ::Tuple{Tuple{Any, Any}, Tuple{Any, Any}}) where {K, V} (0 children)
                 5: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator{_A, F} where {_A, F<:DifferentiationInterface.var"#_prepare_jacobian_aux##0#_prepare_jacobian_aux##1"}, ::Any) where {K, V} (0 children)
                 6: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator{_A, F} where {_A, F<:DifferentiationInterface.var"#_prepare_jacobian_aux##12#_prepare_jacobian_aux##13"}, ::Any) where {K, V} (0 children)
                 7: signature Tuple{typeof(merge!), Any, AbstractDict} triggered MethodInstance for Base.grow_to!(::AbstractDict{K, V}, ::Base.Generator{I, Base.var"#Generator##0#Generator##1"{NonlinearSolveBase.Utils.var"#norm_op##2#norm_op##3"{typeof(+)}}} where I<:(Base.Iterators.Zip{Is} where Is<:Tuple{Base.AbstractBroadcasted, Vector{Float64}}), ::Tuple{Tuple{Any, Any}, Int64}) where {K, V} (0 children)
   backedges: 1: superseding merge!(d::AbstractDict, others::AbstractDict...) @ Base abstractdict.jl:224 with MethodInstance for merge!(::AbstractDict, ::AbstractDict) (1126 children)

inserting merge(d::OrderedCollections.OrderedDict, others::AbstractDict...) @ OrderedCollections ~/.julia/dev/OrderedCollections/src/ordered_dict.jl:488 invalidated:
   mt_backedges: <38 invalidations I deleted because they have enormous types>
   backedges: 1: superseding merge(d::AbstractDict, others::AbstractDict...) @ Base abstractdict.jl:359 with MethodInstance for merge(::AbstractDict, ::Base.Pairs{Symbol, Union{}, Nothing, @NamedTuple{}}) (21 children)
              2: superseding merge(d::AbstractDict, others::AbstractDict...) @ Base abstractdict.jl:359 with MethodInstance for merge(::AbstractDict, ::Base.Pairs{Symbol, _A, Nothing, A} where {_A, A<:NamedTuple}) (2489 children)
```

Partial alternative to JuliaLang#61329. I added `mergewith()` and `mergewith!()`
to be on the safe side.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant