Skip to content

InteractiveUtils: support type annotations as substitutes for values#57909

Merged
JeffBezanson merged 24 commits intoJuliaLang:masterfrom
serenity4:introspection-type-args
Apr 17, 2025
Merged

InteractiveUtils: support type annotations as substitutes for values#57909
JeffBezanson merged 24 commits intoJuliaLang:masterfrom
serenity4:introspection-type-args

Conversation

@serenity4
Copy link
Copy Markdown
Member

@serenity4 serenity4 commented Mar 27, 2025

This PR extends code introspection macros (@which, @code_typed and friends) to recognize type annotations of the form f(1, ::Float64) as types to be forwarded as is to the relevant function. Here are a few examples:

@code_typed f(1, ::Float64)
@code_typed f(1, ::Any, 3, ::String...)
@code_typed f(1, ::Any, 3, ::Vararg{String}) # equivalent to the above
@code_typed ref.x = ::Float64
@code_typed (::Task).scope = ::Any
@code_typed ref[] = ::Float64
@code_typed (::Task).result
@code_typed (::Vector{Int})[::Int] = ::Float64
@code_typed (::typeof(sum))(1, 2) # only works for singleton function types at the moment
@code_typed [2 3 (::Int)] # mind the parentheses, `x ::T` parses as `x::T`
@code_typed sum(::Vector{T}; init = ::T) where {T<:Real}

This has a few advantages:

  • It enables queries with non-concrete argument types, such as @code_typed f(::Any) which will be destructured as code_typed(f, (Any,)). Previously, macros could not express this and users had to rely on using the functional form.
  • Vararg arguments may be expressed as f(::Float64...) or f(::Vararg{Float64}), which was also not expressible previously.
  • It is now possible to skip the construction of values. This is particularly useful for types difficult to construct on the spot, which may be the case when no straightforward constructor exists or when construction involves side effects, such as Task.

This is implemented quite straightforwardly by recursively parsing all ::T annotations and turning them into an intermediate InterpolatedType(T) structure, which is unwrapped as T when the types of the arguments are extracted before calling code_typed.

I also added support for (::typeof(sum))(vec) forms, where a function type annotation is treated as its only instance (if we don't have a singleton type, an error is thrown), making it equivalent to sum(vec). The original motivation was to implement type-annotating callable objects such as (::Returns{Int})(args...), but as this requires changes to the underlying introspection functions I preferred to leave that out. I'd be happy to try adding support for it in a future PR if there are no objections.

@JeffBezanson
Copy link
Copy Markdown
Member

Thank you, I've wanted this for a long time!

I don't really like the implementation method of using a special InterpolatedType type. It should be done by simply mapping the input expression to an output expression, where ::T becomes T and anything else x becomes Typeof(x).

@topolarity
Copy link
Copy Markdown
Member

Does this also support UnionAll signatures like @code_typed accumulate(::Vector{T}, ::T, z) where T<:Integer?

@serenity4
Copy link
Copy Markdown
Member Author

Does this also support UnionAll signatures like @code_typed accumulate(::Vector{T}, ::T, z) where T<:Integer?

It doesn't support matching the same TypeVar to several type parameters, currently only @code_typed accumulate(::Vector{<:Integer}, ::Integer, z) would work. But that's a great idea, and it shouldn't be hard actually if we simply extract any top-level where clauses and feed them alongside the final Tuple{...} type.

@topolarity
Copy link
Copy Markdown
Member

Not to keep tacking on feature requests, but Varargs support would also be very nice (e.g. @code_typed +(::Int...))

@serenity4
Copy link
Copy Markdown
Member Author

That's already there :)

It is not worth the possible interference with macro-defined syntax for :(::) operands
@serenity4
Copy link
Copy Markdown
Member Author

I don't really like the implementation method of using a special InterpolatedType type.

Agreed, especially that it is leaked to the user in error messages when something goes wrong. I removed it.

@serenity4 serenity4 force-pushed the introspection-type-args branch from d205f5f to 17468df Compare March 31, 2025 15:49
This caused more trouble than it is worth, best to
keep the current implementation as is for now.
@serenity4
Copy link
Copy Markdown
Member Author

Does this also support UnionAll signatures like @code_typed accumulate(::Vector{T}, ::T, z) where T<:Integer?

This is now supported.

Because the previous implementation was relying on runtime evaluation to dynamically extract types in certain places, and we now do most of the work when processing the macro, a few other implementation changes were necessary. Hopefully, observable behavior should remain identical as it was before, especially when a few packages use gen_call_with_extracted_types or gen_call_with_extracted_types_and_kwargs (even though it is not marked as public anywhere, and not mentioned in official docs). I tested Cthulhu with this PR and it integrates well:

julia> @descend schedule(::Task)
[ Info: tracking Base
schedule(t::Task) @ Base ~/julia/wip4/base/task.jl:980
980 function schedule(t::Task::Task)::Task
981     # [task] created -scheduled-> wait_time
982     maybe_record_enqueued!(t::Task)
983     enq_work(t::Task)::Task
984 end
[truncated]

I also spotted a bug (reproducible on 1.11.4) where @code_typed fails on broadcasting with kwargs:

julia> @code_typed round.([1.3]; digits = 3)
ERROR: MethodError: no method matching typesof(::Vector{Float64}; digits::Int64)
This method may not support any kwargs.

which requires a fair bit of refactoring, that I leave for a follow-up PR to avoid piling on more things on this one.

This PR should be good for another round of review.

@aviatesk aviatesk self-assigned this Apr 16, 2025
Copy link
Copy Markdown
Member

@topolarity topolarity left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't do a complete read-through yet, but it's looking pretty good to me - Thanks @serenity4 !

return :($(fcn)(Core.kwcall, $tt; $(kws...)))
elseif ex0.head === :call
argtypes = Any[get_typeof(arg) for arg in ex0.args[2:end]]
if ex0.args[1] === :^ && length(ex0.args) >= 3 && isa(ex0.args[3], Int)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is wild that we have to do this here 😅

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah... don't look too closely at this ancient code, you may find other weird things 😅

@aviatesk
Copy link
Copy Markdown
Member

I added a few minor comments on the code style, but I think the fundamental approach is correct. Great job implementing this feature!

@JeffBezanson JeffBezanson merged commit c3e7b1b into JuliaLang:master Apr 17, 2025
7 checks passed
@serenity4 serenity4 deleted the introspection-type-args branch April 17, 2025 21:44
aviatesk pushed a commit that referenced this pull request Apr 26, 2025
…on macros (#58222)

Continuing the work done at #57909, this PR adds support for the
following syntax:
```julia
@code_typed f(some_undef_var::Int) # some_undef_var is ignored, only the annotation is used
@code_typed f(; x::Int) # same here, the name is used but not the value
```
This should allow us to copy and paste signatures found in stacktraces,
such as
```julia
julia> f(x; y = 3) = error()
f (generic function with 1 method)

julia> f(1)
ERROR: 
Stacktrace:
 [1] error()
   @ Base ./error.jl:45
 [2] f(x::Int64; y::Int64)
   @ Main ./REPL[40]:1
 [3] top-level scope
   @ REPL[41]:1
   
julia> asin(-2)
ERROR: DomainError with -2.0:
asin(x) is not defined for |x| > 1.
Stacktrace:
 [1] asin_domain_error(x::Float64)
   @ Base.Math ./special/trig.jl:429
 [2] asin(x::Float64)
   @ Base.Math ./special/trig.jl:443
```

where any function call may be copied and pasted into `@code_typed`,
`@edit` etc as is, provided that the function and argument types are
defined in the active module (i.e. `Main`).

Thanks @topolarity for the idea.

---------

Co-authored-by: Cédric Belmant <cedric.belmant@juliahub.com>
LebedevRI pushed a commit to LebedevRI/julia that referenced this pull request May 2, 2025
…uliaLang#57909)

Extend code introspection macros (`@which`, `@code_typed` and
friends) to recognize type annotations of the form `f(1, ::Float64)` as
types to be forwarded as is to the relevant function.

---------

Co-authored-by: Cédric Belmant <cedric.belmant@juliahub.com>
Co-authored-by: Shuhei Kadowaki <40514306+aviatesk@users.noreply.github.com>
LebedevRI pushed a commit to LebedevRI/julia that referenced this pull request May 2, 2025
…on macros (JuliaLang#58222)

Continuing the work done at JuliaLang#57909, this PR adds support for the
following syntax:
```julia
@code_typed f(some_undef_var::Int) # some_undef_var is ignored, only the annotation is used
@code_typed f(; x::Int) # same here, the name is used but not the value
```
This should allow us to copy and paste signatures found in stacktraces,
such as
```julia
julia> f(x; y = 3) = error()
f (generic function with 1 method)

julia> f(1)
ERROR: 
Stacktrace:
 [1] error()
   @ Base ./error.jl:45
 [2] f(x::Int64; y::Int64)
   @ Main ./REPL[40]:1
 [3] top-level scope
   @ REPL[41]:1
   
julia> asin(-2)
ERROR: DomainError with -2.0:
asin(x) is not defined for |x| > 1.
Stacktrace:
 [1] asin_domain_error(x::Float64)
   @ Base.Math ./special/trig.jl:429
 [2] asin(x::Float64)
   @ Base.Math ./special/trig.jl:443
```

where any function call may be copied and pasted into `@code_typed`,
`@edit` etc as is, provided that the function and argument types are
defined in the active module (i.e. `Main`).

Thanks @topolarity for the idea.

---------

Co-authored-by: Cédric Belmant <cedric.belmant@juliahub.com>
topolarity pushed a commit that referenced this pull request Jun 4, 2025
…rospection macros (#58349)

This PR includes a fix and a feature for code introspection macros
(`@code_typed`, `@code_llvm` and friends *but not* `@which`, `@edit`,
etc):
- Fixes a bug for expressions of the form `f.(x; y = 3)`, for which
keyword arguments were not properly handled and led to an internal
error.
- Adds support for broadcasting assignments of the form `x .= f(y)`, `x
.<<= f.(y, z)`, etc.

The way this was (and still is) implemented is by constructing a
temporary function, `f(x1, x2, x3, ...) = <body>` and feeding that to
code introspection functions. This trick doesn't apply to `@which` and
`@edit`, which need to map to a single function call (we could arguably
choose to target `materialize`/`materialize!`, but this behavior could
be a bit surprising and difficult to support).

The switch differentiating the families of macro
`@code_typed`/`@code_llvm` and `@which`/`@edit` etc was further exposed
as an additional argument to `gen_call_with_extracted_types` and
`gen_call_with_extracted_types_and_kwargs`, which default to the
previous behavior (differentiating them based on whether their name
starts with `code_`). The intent is to allow other macros such as
`Cthulhu.@descend` to register themselves as code introspection macros.
Quick tests indicate that it works as intended, e.g. with this PR
Cthulhu supports `@descend [1, 2] .+= [2, 3]` (or equivalently, as added
in #57909, `@descend ::Vector{Int} .+= ::Vector{Int}`).

I originally just went for the fix, and after some refactoring I
realized the feature was very straightforward to implement.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants