Generic #[wasm_bindgen] imports via per-monomorphisation#5180
Draft
guybedford wants to merge 37 commits into
Draft
Generic #[wasm_bindgen] imports via per-monomorphisation#5180guybedford wants to merge 37 commits into
guybedford wants to merge 37 commits into
Conversation
…mission Bare-minimal first milestone for proper (non-type-erased) generics on imported functions. Adds an opt-in `generic` attribute that, instead of the existing ErasableGeneric single-shim erasure, emits a per-(import, T) call-site courier modelled on wbg_cast. For `#[wasm_bindgen(generic)] fn log<T>(x: T)` each monomorphisation deposits, as i32.const immediates, the JS import name and the concrete argument schema, then calls the marker import `__wbindgen_describe_generic_import`. The cli (milestone 2) will scan these, synthesise a per-T JS adapter bound to the named import, and rewrite the courier call sites. Supported shape so far (bails otherwise): free function, one type parameter, one owned argument of that parameter, unit return. Verified emission: three calls log_generic(u32/f64/&str) produce three distinct couriers with native per-T ABI signatures ((param i32), (param f64), (param i32 i32)), one shared name pointer, and three distinct schema pointers.
…orts Milestone 2: the cli side of per-monomorphisation generic imports. Generalises the closure-cast structural scanner to an arbitrary marker arity and adds execute_generic_imports, a sibling of execute_casts that scans __wbindgen_describe_generic_import calls, reads the JS import name and concrete argument schema from the data section, and composes a function descriptor [FUNCTION, 0, 1, <arg>, UNIT, UNIT]. In wit processing, each recovered (name, T) synthesises an import bound to AuxImport::Value(Bare(determine_import(name))) so the adapter *calls* the named JS function (unlike casts, which are identity), then rewrites the courier call sites and GCs the scaffolding. The marker import is skipped in verify like __wbindgen_describe_cast. Verified end to end: log_generic(u32/f64/&str) produces three distinct JS adapters with correct per-T marshalling — f64 direct, u32 unsigned coercion, and &str decoded via getStringFromWasm0(ptr,len) — all calling the real JS log_generic. The string case is impossible under the type-erased ErasableGeneric path. Cast path and cli-support tests unaffected (65 passed).
…orts Adds an end-to-end runtime test (tests/wasm/generic_import.rs) that invokes a generic import with u32/f64/&str and asserts JS receives the correctly-marshalled values — proving per-monomorphisation marshalling works at runtime, not just in codegen snapshots. The &str case (two-word ABI decoded to a JS string) is impossible under the type-erased path. To make the import bind to a real JS module, threads the import module from the macro through the courier to the cli: - capture js_module on ast::ImportFunction at parse time; - carry it (plus macro-computed NAME_LEN/MODULE_LEN consts, so lengths fold to i32.const instead of debug-only str::len calls that would reset the cli scanner) through GenericImportName + the marker (now 6 immediates: name ptr/len, module ptr/len, schema ptr/len); - the cli reads the module and binds via determine_import. Known limitation (follow-up): the synthesised import uses RawNamed (plain module specifier) rather than Named, because the generic call-site path does not yet register a linked module, so Named would resolve to a missing ./snippets/<module> entry. Proper linked-module registration is deferred.
Introduces the shared-template / per-variant-fill model. Adds a TYPE_PARAM(i) schema opcode marking generic-parameter holes. The macro emits a per-import signature template (a full function descriptor with TYPE_PARAM holes) as an addressable const on the import's name carrier; each monomorphisation's courier deposits the template pointer plus the concrete per-T fill schema. The cli splices fills into the template's holes to recover the concrete descriptor. This replaces the cli composing the descriptor from a bare arg schema with template+splice, establishing the mechanism that lets one shared template back all monomorphisations. Metadata routing through import_function and dropping the name/module immediates follow next. Runtime test still green.
Moves the JS name and module into the shared template const as a small char-per-word metadata header (flags, name, module) preceding the holed signature. The courier marker shrinks to its final compact form (template_ptr, template_len, fill_ptr, fill_len): the template pointer is now the single shared handle serving as dedup key, retention anchor, and read source, with only the per-T fill varying per call site. The cli parses the header (reusing the [count, chars..] string convention of the descriptor decoder) and splices the signature. Pure refactor; the runtime test still passes. Namespace/catch/variadic and routing through import_function follow.
…oding Reworks the generic-import path so import metadata (js_name, module, js_namespace, catch, variadic) comes from the normal AST custom section keyed by shim name — the same source every other import uses — instead of being re-encoded into the descriptor const. The carrier const now holds only the shim key + the holed signature template; the courier marker carries (shim, template, fill). The cli collects per-monomorphisation (shim -> [(courier, concrete descriptor)]) from the descriptor section, captures each generic import's AST metadata during the program loop (import_function), then binds each distinct descriptor after the loop via determine_import + import_adapter + catch/variadic flags — the exact machinery normal imports use. Monos whose T resolves to the same schema collapse to one import. This removes the re-encoded name/module header and the RawNamed module shortcut: the test now resolves its module through the proper AST path (Named), and DescribeImport skips emitting a stray erased descriptor for generic imports. Establishes the reference chain fills -> descriptor (holed template) -> AST. Runtime test green; casts and 65 cli-support tests unaffected. ast::ImportFunction.js_module (added only for the bespoke header) is removed.
…Param Replace the flat word-scan splice with a tree substitution on the decoded descriptor: templates decode to a Descriptor carrying TypeParam(idx) holes, fills decode to descriptors, and substitute() walks the tree replacing holes. Correct by construction once concrete argument schemas (which can embed words equal to the TYPE_PARAM opcode) appear in templates.
…ed-arity couriers
Replace the single-arg/unit-return generic import courier with a
fixed-arity courier family (rt) and a macro path that builds an
arbitrary holed signature template from the argument and return types.
- rt: __WbgTypeParam<I> hole marker (WasmDescribe -> [TYPE_PARAM, I]),
GenericFills{1..8} fills carriers (concatenated per-param schemas), and
wbg_generic_import_{0..8} courier family (per arg arity), each splitting
every argument's ABI into prims with the black_box + keep_prims_alive
anti-DCE treatment and returning WasmRet<R::Abi>.
- macro: build the template by substituting type params with __WbgTypeParam
holes and reusing the normal WasmDescribe schema construction, so nesting
(&_, Option<_>, ...) holes correctly; emit the fills carrier from the
distinct params and dispatch to the right-arity courier.
- cli: fills transport is now a sequence of self-delimiting descriptors
(decode_sequence), indexed by type-parameter position for substitute().
Supports multi-arg, mixed concrete/generic args, repeated params, multiple
distinct type params, concrete + generic returns, zero-arg generic return,
and Option<T> nesting. Refs/lifetimes, async/catch/variadic, and methods
remain deferred.
- catch: declared return stays Result<T, JsValue>; courier returns the inner Ok type and the wrapper runs take_last_exception() + wraps in Ok. Binding already flags catch on the synthesised adapter. - references: hole_type pins template refs to 'static (carrier const has no lifetime); the wrapper introduces a synthetic 'wbg_a lifetime for elided refs so &T arguments are valid in turbofish/where-bound position. - variadic is no longer rejected (binding flags it); async stays deferred.
… param Generalises the signature template: any argument/return type that mentions a type parameter becomes one TYPE_PARAM hole, filled per-monomorphisation by that whole type's WasmDescribe schema (deduped by type). Parameter-free types contribute their schema inline. The fills carrier is parameterised by the distinct hole types rather than the bare params. This removes the __WbgTypeParam marker and, crucially, handles holes that aren't bare idents (e.g. associated-type returns like <T as Trait>::Assoc), which the method path needs.
bind_generic_imports now mirrors import_function's method dispatch: capture owned method data (class + operation kind) and structural into GenericImportMeta, and select the adapter kind + JS-import shape via determine_import_op. determine_import_op now takes the function name as &str so it can be reused from the generic binding path.
Route generic methods through get_fn_generics for class-generic hoisting and emit the wrapper inside the class impl, carrying the receiver as the courier's first argument (with the synthetic 'wbg_a lifetime on &self). - instance methods on concrete classes (runtime-tested: per-T marshalling through one bound JS method), - generic classes with class-generic hoisting and projection holes like <T as Trait>::Assoc in return position (compile-validated, DurableObject shape), - constructors/static/getters/setters dispatch via the cli binding landed earlier. Holes are keyed by parameter-dependent type, so the receiver &Class<T> and an associated-type return each become their own fill.
Declare and instantiate a #[wasm_bindgen(generic)] import only in dependency crate a, then call it from the main test crate. Verifies the per-monomorphisation courier + holed-template rodata survive the archive pull into the final artifact, in both debug and release. No descriptor anchor is needed — the rt-hosted courier retains like casts do.
…ke shims Generic closure import args (&dyn Fn / &mut dyn FnMut whose signature mentions a type parameter) now route through a per-T invoke shim instead of the fixed-export-name wrapper: - rt: GenericFill trait (blanket for describable holes; ClosureFill<C, MUT> for closure holes) supplies each hole's schema plus, for closures, the WasmClosure::invoke_shim_addr. GenericFills exposes invoke_addr; the courier marker gains a 7th immediate carrying the single closure hole's invoke-shim table index. All invoke_addr forwarding is #[inline(always)] so it folds to an i32.const for the scanner. - macro: closure args that mention a type parameter become a ClosureFill hole; the closure type C is bound WasmClosure + WasmDescribe. - cli: scanner reads the 7th immediate and patches the closure descriptor's shim_idx placeholder (Descriptor::patch_closure_shim), mirroring the closure-cast compose_cast_descriptor path. Tested: FnMut(T), FnMut(Option<T>), and FnMut() -> T, each marshalling per concrete T.
Confirms vector-of-generic holes: Vec<T> is one hole whose fill resolves per concrete T (typed-array marshalling for Vec<u32>).
JsGeneric now also requires IntoWasmAbi + FromWasmAbi + WasmDescribe (it already did, transitively, for every implementor). This lets generic imports converted to the per-monomorphisation path keep their existing `T: JsGeneric` bounds with no caller churn: the bound now directly satisfies the per-mono path's ABI requirements. ErasableGeneric remains a supertrait transitionally so un-converted erasure methods keep compiling; it is dropped once every method is migrated.
The per-monomorphisation path dropped the function's rust_attrs, so cfg-gated method variants (e.g. js-sys stable vs unstable `at`/`get`/`set`) both emitted, producing duplicate definitions. Emit #(#attrs)* on the carrier struct, the GenericImportName impl, and the wrapper fn so cfg gates them together. js-sys: convert the Array<T> accessor cluster (new/new_typed/ new_with_length(_typed), at, get, get_unchecked, get_checked, set, set_ref, delete) to the per-monomorphisation path. Their existing JsGeneric/unbounded T bounds satisfy the per-mono ABI requirements with no caller churn.
Proves multi-argument generic closures (FnMut(T, u32, Array<T>)) work through the per-mono path in real js-sys, with no caller churn.
… a sum A generic import with multiple holes (e.g. a method: receiver + closure) summed each hole's invoke addr in GenericFills::invoke_addr; `0 + addr` lowers to an i32.add the cli scanner cannot fold, so it read 0 and left the closure shim_idx placeholder unpatched (panicking with "failed to find 0 in function table"). Move the address to a dedicated ClosureInvoke carrier (NoClosure / ClosureFill<C, MUT>) passed as its own courier type param, so it stays a lone i32.const. At most one closure argument per generic import. js-sys: convert Array every/for_each (multi-arg generic closures), now runtime-verified against the Array test suite.
…enerics Add generic to the Array<T> methods (accessors, search/iteration closures, map/reduce/filter/find/sort families, slice/splice/concat, etc.), routing them through the per-mono path. Their existing T: JsGeneric / unbounded-T bounds satisfy the per-mono ABI requirements via JsGeneric's new ABI supertraits, with no caller churn. Deferred (kept on the erasure path for now): the from*/fromAsync Iterable/ AsyncIterable constructors (async + trait-bound element types) and push (its generic internal caller extend_typed needs &T: IntoWasmAbi, pending a JsGeneric borrow supertrait). Array + ArrayIterator suites pass.
push<T>(value: &T) needs &T: IntoWasmAbi for its generic internal callers (from_iter_typed/extend_typed). Adding for<'a> &'a Self: IntoWasmAbi to JsGeneric explodes across every T: JsGeneric site, so instead require for<'a> &'a T: IntoWasmAbi only on those two methods, which is where the generic push call actually occurs.
Iterator<T>::next, AsyncIterator<T>::next/next_iterator route through the per-mono path (T appears in the IteratorNext<T> / Promise<IteratorNext<T>> return hole). Iterator suites pass.
Two coordinated changes so generic imports introduce no synthetic lifetime (which previously broke higher-ranked fn-item use of generic getters): - rt: the courier takes N/F/IC/R as zero-sized Tag<_> *values*, so the macro calls it with no turbofish; every type parameter (including the ABI arg types) is inferred from the call's values, leaving argument references with their elided, higher-ranked lifetimes. Tag is zero-sized, so the wasm signature is unchanged. - macro: parameter-dependent references are decomposed structurally in the template (REF/REFMUT opcode + owned-leaf hole), so every fills-carrier hole type is owned and lifetime-free; reference arguments get a higher-ranked IntoWasmAbi/WasmDescribe bound. The receiver is plain &self again. No behaviour change under the opt-in (generic_import 14, js-sys Array 224 still pass); this unblocks the global default-on switch.
Route every eligible import function/method through the per-monomorphisation path automatically (generic_eligible), with the legacy erasure path as a fallback for what per-mono can't yet express (async, explicit lifetimes, const generics, >8 args, >8 holes, >1 generic closure, ScopedClosure args, or an unused type parameter). The #[wasm_bindgen(generic)] opt-in still forces the path. cli: fix inline-JS module misclassification for generic imports. Inline snippet module indices are program-local and depend on unique_crate_identifier, which is only correct during the owning program's pass; bind_generic_imports runs afterwards. Capture the resolved (crate, snippet index) identity at capture time (OwnedImportModule:: InlineResolved) and resolve it directly at bind time, so an inline-JS generic import no longer binds to a different crate's snippet. macro: carry only cfg attrs onto the carrier struct/impl (not #[deprecated] etc.). js-sys: localized for<'a> &'a T: IntoWasmAbi bounds on the generic callers that pass &T (callN args, ArrayTuple::new, new_value, futures). Green under the flip: wasm-bindgen 393, js-sys 738, web-sys builds.
Async (`async fn`) generic imports now route through the per-mono path: the courier returns the imported Promise<inner> (externref ABI), and the generated async wrapper awaits it via a typed JsFuture<inner>, surfacing the per-T resolution value (with catch -> Result and unit handling). The inner success type is bound FromWasmAbi + 'static for the JsFuture await; generic_eligible no longer excludes async. js-sys: JsFuture<T> no longer implements ErasableGeneric — its T is purely Rust-side typing resolved through the typed From<Promise<T>>, so the erasure impl was vestigial. Tested: async_echo<T>(x: T) -> T resolves per T (u32, String); wasm-bindgen 394, js-sys 738, web-sys build, futures/async_vecs/Promise suites green.
Extend the courier family (wbg_generic_import_9..11) and fills carriers (GenericFills9..12), and raise generic_eligible / the per-mono codegen caps to 11 arguments and 12 distinct holes. This converts the remaining high-arity generic imports to per-mono: Function::call7..9 / bind7..9 (receiver + context + up to 9 args) and ArrayTuple::new8 (9 holes), which previously fell back to the erasure path. Function (27) + ArrayTuple (25) suites pass. Remaining erasure holdouts: ScopedClosure-argument methods (Promise then/catch/finally/* and Array::from_async_map).
…mono is now the sole generic-import path - allow explicit lifetime parameters (get_fn_generics already threads fn lifetimes into the wrapper impl generics), so Promise then/catch/finally and Array::from_async_map (which carry a 'a scope lifetime) convert. - only emit the per-argument ABI bound for parameter-dependent arguments; concrete arguments are proven from their own type at the courier call, and a concrete reference may carry an elided lifetime parameter (e.g. &ScopedClosure<dyn FnMut(JsValue)>) that can't be named in a bound. With this every generic import in js-sys, web-sys, and the wasm-bindgen test crate routes through per-monomorphisation (verified: 0 erasure fallbacks). Suites green: js-sys 738, wasm-bindgen 394, web-sys builds.
With per-monomorphisation the sole path for type-parameter generic imports, the erasure branches in the normal import codegen are dead. Rip them out: - delete the ErasableGenericOwn/Borrow/Mut argument and return bounds plus the generic_to_concrete + transmute handling in ImportFunction:: try_to_tokens and DescribeImport. - delete the now-unused erasure helpers: generic_to_concrete, GenericRenameVisitor, generic_param_names, all_param_names, uses_lifetime_params(unused dup), and the FnClassGenerics concrete_defaults field + add_fn_bound, and their tests. Lifetime-only imports (no type parameters, e.g. &ScopedClosure<'a, dyn FnMut(..)>) still need their wrapper lifetimes pinned to 'static for the extern ABI (an extern block can't reference outer lifetimes); that staticize + transmute handling — previously entangled with the erasure branch — is preserved as a dedicated lifetime-only branch. The ErasableGeneric trait marker itself is intentionally kept for now. Green: wasm-bindgen 394, js-sys 738, web-sys builds, closures 66.
Per-monomorphisation is now the default for every eligible generic import (generic_eligible), so the explicit opt-in is redundant. Remove the `generic` attribute from the attribute table, parser, and the ast::ImportFunction field, and drop the `self.generic ||` / `f.generic ||` shortcuts in the codegen gates (now just generic_eligible()). Strip the now-defunct attribute from all call sites: js-sys (88 lines), the wasm-bindgen generic-import tests, and the cross-crate test crate; they continue to route through per-mono via eligibility unchanged. Green: wasm-bindgen 394, js-sys 738, web-sys builds, generic_import 15.
The ErasableGenericOwn/Borrow/BorrowMut marker traits only served the
removed type-erasure import codegen; they are now unreferenced. Drop them,
keeping the core ErasableGeneric (type Repr) substrate for zero-cost
upcast() and slice/vector ABI.
The generic-{import,return}-non-erasable ui-tests asserted that passing a
non-erasable type (u32) errored; per-monomorphisation now marshals such
types directly, so those tests are obsolete.
Option<T> gains VectorIntoWasmAbi / VectorFromWasmAbi / WasmDescribeVector impls following the same composite-element path as Vec<String> and Vec<JsValue>: each element converts to a JsValue (None -> undefined via the existing JsValue: From<Option<T>>), transferred as an externref index buffer with descriptor VECTOR, EXTERNREF. Option<T> has no fixed scalar layout, so it cannot be packed into a typed-array buffer - the externref array is the only viable wire shape. No cli-support changes: the externref-array shim already handles this descriptor. Works both non-generically and through the per-mono generic path (whose fill carries the concrete Vec<Option<T>> schema).
JsGeneric carried IntoWasmAbi + FromWasmAbi + WasmDescribe as scaffolding for the removed type-erasure path, which forced every generic element to be representable in both directions even where a position uses only one. Per-monomorphisation supplies the single direction it needs on the concrete type at the call site, so those supertraits are now vestigial. Narrow JsGeneric to the handle identity (ErasableGeneric<Repr = JsValue> + Upcast + JsCast + 'static) and add the explicit single-direction bound at each hand-written js-sys site that relied on the supertrait: Function call/bind tuples, ArrayTuple new/get, the Array iterators + IntoIterator, Array::push, and PropertyDescriptor::new_value. Bound-neutral: every JS handle already implements the ABI traits, so no type that worked before is excluded.
Add runtime cases exercising per-monomorphisation paths that were previously uncovered: * primitive ABIs beyond u32/f64/&str (bool, i64, char) * high arity (nine generic args -> wbg_generic_import_9 courier) * generic static method, generic setter, generic constructor on a generic class (instantiated, not compile-only) * two-parameter generic closure FnMut(A, B) * zero-copy JS-handle Vec<T> (ErasableGeneric slice path) * nested Vec<Option<T>> as a (T | undefined)[] externref array
A generic import taking `&U` (e.g. `Promise::resolve<U: Promising>(obj: &U)`) could not be instantiated with a primitive `U` like `u32`: `&u32` had no `IntoWasmAbi` impl, so `Promise::<u32>::resolve(&5)` failed to compile. Two layers: * core: blanket `impl<T: Copy + IntoWasmAbi> IntoWasmAbi for &T` passing the value by copy. Coherent with the existing `&Handle`/`&str`/`&[T]` ref impls because those referents are not `Copy`. * cli-support: `outgoing_ref` now treats `Ref(<primitive>)` like the by-value primitive — the wire already carries the copied value, so JS just receives the number/bool/char. Enables `Promise<u32>` and other `fn(&T)` imports for value-type `T`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #5171.
Implements generic
#[wasm_bindgen]imports via per-monomorphisation: a generic imported function, method, constructor, getter, setter or static emits a dedicated binding for each concrete instantiation at the call site, marshalling the type parameter by its own ABI.Generic imports accept the full range of ABI types — primitives, refs/lifetimes,
Option<T>,Vec<T>,Vec<Option<T>>(as a(T | undefined)[]externref array), generic closures,asyncandcatch.&Targuments also work for copyable value types, sofn(&U)imports likePromise::resolveacceptPromise<u32>.JsGenericis the JS-handle identity bound (ErasableGeneric<Repr = JsValue>+ upcast/cast); it carries no ABI supertraits, with explicit single-direction bounds at the hand-written js-sys sites that need them.Draft — opening early for review of the approach.