[Impeller] Namespace user-supplied shaders to prevent entrypoint collisions#186332
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a namespacing mechanism for user-supplied shaders to prevent collisions with engine-internal entrypoints by adding a library_id to RuntimeStage and Shader objects. Feedback focuses on performance and maintainability, specifically suggesting the caching of scoped names to avoid per-frame allocations, refactoring duplicated fallback ID generation logic, and using more idiomatic string appending.
|
Ah yeah that should mostly resolve the user<->engine case. However, a deeper motivation for this change is actually to namespace user shaders against each other. Will retitle the PR. Concretely, this prevents:
|
| std::string ShaderKey::MakeUserScopedName(std::string_view scope, | ||
| std::string_view library_id, | ||
| std::string_view entrypoint) { | ||
| std::string out; |
There was a problem hiding this comment.
absl::StrCat please (better performance and ergonomics)
| ShaderKey::kScopeFlutterGPU, | ||
| "packages/pkg_a/assets/shaders.shaderbundle", | ||
| "solid_fill_fragment_main"), | ||
| "fg:packages/pkg_a/assets/shaders.shaderbundle:solid_fill_fragment_main"); |
There was a problem hiding this comment.
how long can we make these before we get a problem?
There was a problem hiding this comment.
Done in 09272b8: new ShaderKeyTest.MakeUserScopedNameHandlesLongInputs covers a 4096-byte library_id and a 2048-byte entrypoint. The scoped name is only used as a std::string map key, so there is no fixed length limit; this just pins that expectation.
| std::shared_ptr<const impeller::ShaderFunction> Shader::GetFunctionFromLibrary( | ||
| impeller::ShaderLibrary& library) { | ||
| return library.GetFunction(entrypoint_, stage_); | ||
| return library.GetFunction(GetScopedName(), stage_); |
There was a problem hiding this comment.
I don't think there is a test exercising this, is there?
There was a problem hiding this comment.
Done in 3617309: added Two shader bundles with the same entrypoint names do not collide in gpu_test.dart, which loads test.shaderbundle and a new test_alt.shaderbundle (both exporting UnlitFragment / UnlitVertex) and renders with a RenderPipeline from each.
| // same asset evicts the previous registration at the same scoped key, | ||
| // and so two different FragmentProgram assets cannot collide with each | ||
| // other or with engine-internal entrypoint names. | ||
| runtime_stage->SetLibraryId(asset_name); |
There was a problem hiding this comment.
I don't think there is a test for this either.
There was a problem hiding this comment.
Done in 3617309: added FragmentPrograms from different asset paths do not collide in fragment_shader_test.dart, which loads no_uniforms.frag.iplr and a byte-identical alias no_uniforms_alt.frag.iplr and paints a Picture with each.
|
autosubmit label was removed for flutter/flutter/186332, because - The status or check suite Linux linux_unopt has failed. Please fix the issues identified (or deflake) before re-applying this label. |
… engine-internal shaders Flutter GPU user shaders, FragmentProgram (RuntimeEffect) shaders, and Impeller's own entity-contents shaders all register into the same per-Context impeller::ShaderLibrary, keyed only on (entrypoint name, stage). Because impellerc derives entrypoint names from source file stems with no scope prefix, a user-supplied shader whose .frag stem matches an Impeller-internal one would either shadow it on lookup or, in the FragmentProgram case (which intentionally evicts the previous function on dirty re-load to support hot reload), actively unregister the engine's pipeline function. Add a small ShaderKey::MakeUserScopedName helper that prefixes user shader entrypoints with "<scope>:<library_id>:" before they go into the shared registry. Engine-internal entrypoints stay unprefixed and are not spoofable, since impellerc-generated names are valid identifiers and cannot contain ':'. The library_id is anchored to the asset path the shader was loaded from (FragmentProgram via SetLibraryId on RuntimeStage; Flutter GPU via the asset name plumbed through ShaderLibrary::MakeFromFlatbuffer), so hot reload of the same asset continues to evict and replace the same registry slot, and shaders loaded from different packages or different asset bundles never collide. Fixes flutter#175464.
…k id, tighten append - Cache the scoped fragment registry name on RuntimeEffectContents at SetRuntimeStage time so that the per-frame Render path no longer re-allocates and re-concatenates it on every call. - Move the fallback library-id helper (used when no asset path is available) into ShaderKey::MakeFallbackLibraryId so the runtime stage decode path and the Flutter GPU bundle path share one source of truth and one process-wide counter, and use std::to_string instead of std::ostringstream. - Tighten ShaderKey::MakeUserScopedName to use std::string::append with std::string_view directly.
impeller/renderer (where ShaderKey lives) already depends on impeller/runtime_stage, so runtime_stage.cc cannot pull in ShaderKey::MakeFallbackLibraryId without creating a cycle. Restore a TU-local helper in runtime_stage.cc and document why; flutter::gpu continues to use the shared ShaderKey helper since lib/gpu already depends on impeller/renderer. Both helpers use std::to_string per the earlier review feedback.
* `ShaderKey::MakeUserScopedName` and `ShaderKey::MakeFallbackLibraryId` now use `absl::StrCat` instead of the manual `reserve` / `append` / `push_back` pattern. Improves readability and matches the prevailing style used elsewhere in Impeller. * Adds `ShaderKeyTest.MakeUserScopedNameHandlesLongInputs` covering a 4096-byte `library_id` and a 2048-byte `entrypoint`. The scoped name is only used as a `std::string` key in the per-process shader library registry, so there is no fixed length limit; this pins that expectation against future regressions. * Adds the `absl::strings` dependency to `impeller/renderer/BUILD.gn` for `absl::StrCat`.
…tPrograms Adds Dart-level integration tests that pin the user-visible behavior the shader namespacing change is supposed to deliver: two distinct user shader sources that share an embedded entrypoint name can be loaded and used in the same process without colliding in the shared shader library. * `testing/dart/gpu_test.dart`: new `Two shader bundles with the same entrypoint names do not collide` test that loads `test.shaderbundle` and a new `test_alt.shaderbundle` (both exporting `UnlitFragment` and `UnlitVertex`), constructs a `RenderPipeline` from each, and renders with both. Pre-namespacing this would have torn one of the two pipelines down at registration time. * `lib/ui/fixtures/shaders/BUILD.gn`: new `flutter_gpu_shaders_alt` impellerc target that produces `test_alt.shaderbundle` with the same `UnlitFragment` / `UnlitVertex` entrypoints as the existing bundle. * `testing/dart/fragment_shader_test.dart`: new `FragmentPrograms from different asset paths do not collide` test that loads `no_uniforms.frag.iplr` and a byte-identical alias `no_uniforms_alt.frag.iplr`, gets a `FragmentShader` from each, and paints a Picture with both. `no_uniforms.frag` was chosen so the test does not need to thread sampler or uniform setup through the FragmentShader to validate it. * `lib/ui/fixtures/shaders/general_shaders/BUILD.gn`: new `no_uniforms_alt` copy action that publishes `no_uniforms.frag.iplr` under a second asset name so FragmentProgram tracks the two loads under distinct library ids. Verified locally on both `flutter_tester` (Metal) and `flutter_tester_opengles` (SwANGLE). The unrelated `impellerc produces reasonable JSON encoded IPLR files` failure in `fragment_shader_test.dart` is pre-existing on master.
…sion test CI analyzer flagged `omit_obvious_local_variable_types` on four locals in the new `FragmentPrograms from different asset paths do not collide` test where the right-hand side is a constructor call (`Paint()`, `PictureRecorder()`, `Canvas(...)`) or a typed list literal (loop variable). Drop the annotations so the lint passes.
The prior fix for `omit_obvious_local_variable_types` overshot by also dropping the type annotation on `final picture = recorder.endRecording()` at fragment_shader_test.dart:88. `PictureRecorder.endRecording()` is a method call, not a constructor, so the analyzer considers the type non-obvious and the `specify_nonobvious_local_variable_types` lint fired on the next CI run. Restore `final Picture picture = ...` to keep the annotation on the function-call return while leaving the constructor locals (`Paint paint = Paint()`, `PictureRecorder recorder = ...`, `Canvas canvas = Canvas(...)`) without explicit types. Verified locally: `engine/src/flutter/ci/analyze.sh` reports "No issues found!".
85d99ba to
139c8fe
Compare
…ookup
`ShaderLibraryMTL::GetFunction(name, stage)` falls back to a per-library
`[MTLLibrary newFunctionWithName:]` lookup that requires the registered
name to match the MSL function symbol. Namespaced registrations from the
shader namespacing change (e.g. `re:<library_id>:<entry>` from
`RuntimeEffectContents`) never match, so `GetFunction` returned null
immediately after a successful `RegisterFunction`. This surfaced as
`EntityTest.RuntimeEffectCanSuccessfullyRender/{Metal,MetalSDF}` failing
on master with "Failed to fetch runtime effect function immediately
after registering it".
Populate the `functions_` cache at `RegisterFunction` time: after the
new `MTLLibrary` compiles, find the function whose `functionType`
matches the requested stage and cache it under the registration name.
`GetFunction`'s pre-existing cache check at the top of the method then
short-circuits the library-iteration fallback, so namespaced names
resolve to the right function.
`UnregisterFunction` gets a matching cache-first path that retrieves the
target `MTLLibrary` from the cached `ShaderFunctionMTL::library_`
(accessible via the existing `ShaderLibraryMTL` friend), removes it
from `libraries_`, and evicts the cache entry. The pre-existing
library-iteration fallback is kept for engine-bundled libraries
registered via the multi-library constructor (which never go through
`RegisterFunction` and so are not in the cache).
Also promote `UnregisterFunction`'s outer lock from `ReaderLock` to
`WriterLock`, since both the cache-first path and the fallback mutate
state (`libraries_` and `functions_`).
Verified locally on the previously-failing tests (Metal, MetalSDF, GLES
variants of `RuntimeEffectCanSuccessfullyRender` and
`RuntimeEffectCanPrecache` all pass; Vulkan skipped because no MoltenVK
on the test machine). The broader `*RuntimeEffect*`, `*ShaderLibrary*`,
`*ShaderKey*` filter reports 29 passing. `gpu_test.dart` still passes
on `flutter_tester_opengles` and `flutter_tester` (51 + 1 skipped).
|
This time for sure. 😆 |
…11713) Manual roll requested by bmparr@google.com flutter/flutter@23f6f58...0541913 2026-05-14 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[Windows] Propagate the enabled accessibility state (#184501)" (flutter/flutter#186492) 2026-05-13 srawlins@google.com [dev] Use super parameters in missed spots (flutter/flutter#186193) 2026-05-13 loic.peron@inetum.com [Windows] Propagate the enabled accessibility state (flutter/flutter#184501) 2026-05-13 matt.boetger@gmail.com [flutter_tool] filter out MotionEvent-JNI warning spam from logcat (#174783) (flutter/flutter#186079) 2026-05-13 engine-flutter-autoroll@skia.org Roll Packages from 93cbed6 to 2ec2236 (1 revision) (flutter/flutter#186464) 2026-05-13 mdebbar@google.com [web] Fix untriaged issues link label (flutter/flutter#186465) 2026-05-13 bdero@google.com [Impeller] Namespace user-supplied shaders to prevent entrypoint collisions (flutter/flutter#186332) 2026-05-13 1063596+reidbaker@users.noreply.github.com [flutter_tools] Migrate detectLowCompileSdkVersionOrNdkVersion to AGP task (flutter/flutter#184731) 2026-05-13 jason-simmons@users.noreply.github.com Update the Flutter Gallery web app template files to support running with Wasm (flutter/flutter#186268) 2026-05-13 jason-simmons@users.noreply.github.com [web] Use heap allocation for buffers that would consume too much space on the Wasm stack (flutter/flutter#186228) 2026-05-13 engine-flutter-autoroll@skia.org Roll Skia from 56ca5896c0d9 to 27f7bba22600 (3 revisions) (flutter/flutter#186444) 2026-05-13 engine-flutter-autoroll@skia.org Roll Fuchsia Linux SDK from z7ICmPtn4hspu02zk... to y6uQHA5xUN83IF395... (flutter/flutter#186442) 2026-05-13 engine-flutter-autoroll@skia.org Roll Skia from 6385958d2feb to 56ca5896c0d9 (1 revision) (flutter/flutter#186441) 2026-05-13 engine-flutter-autoroll@skia.org Roll Dart SDK from 9576691c37d8 to 8e30b88e4d5a (1 revision) (flutter/flutter#186429) 2026-05-13 engine-flutter-autoroll@skia.org Roll Skia from 77a21bc723dc to 6385958d2feb (9 revisions) (flutter/flutter#186428) 2026-05-13 164032450+AlexEduV@users.noreply.github.com Docs/improving docs for semantics UI lib (flutter/flutter#186125) 2026-05-12 jason-simmons@users.noreply.github.com [Tool] Support glob patterns when parsing workspaces in FlutterProject (flutter/flutter#185715) 2026-05-12 nico.reiab@gmail.com docs: fix overriden -> overridden in MediaQueryData dartdoc (flutter/flutter#186323) 2026-05-12 brackenavaron@gmail.com [Test cross imports] No material in `test/foundation`, `test/gestures`, `test/semantics`, `test/services` (flutter/flutter#186144) 2026-05-12 nico.reiab@gmail.com docs: fix "tha" -> "that" typo in widget_inspector_test comment (flutter/flutter#186322) 2026-05-12 nico.reiab@gmail.com docs: Fix doubled-word typos in framework dartdoc (flutter/flutter#186319) 2026-05-12 137456488+flutter-pub-roller-bot@users.noreply.github.com Roll pub packages (flutter/flutter#186418) 2026-05-12 30870216+gaaclarke@users.noreply.github.com Bumped required mediatek vender sdk version. (flutter/flutter#186405) 2026-05-12 magder@google.com Make DeepLinkJsonFromManifestTask Gradle task build cacheable (flutter/flutter#185903) 2026-05-12 66727653+ishaq2321@users.noreply.github.com Harden dev tooling scripts against command injection and log leaks (flutter/flutter#186076) 2026-05-12 137456488+flutter-pub-roller-bot@users.noreply.github.com Roll pub packages (flutter/flutter#186274) 2026-05-12 bdero@google.com [Flutter GPU] Allow allocating multi-mip textures and overwriting specific (mip, slice) levels (flutter/flutter#185890) 2026-05-12 zhongliu88889@gmail.com [web] Fix MenuAnchor dismiss when semantics enabled (flutter/flutter#183093) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC bmparr@google.com,stuartmorgan@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
…lutter#11713) Manual roll requested by bmparr@google.com flutter/flutter@23f6f58...0541913 2026-05-14 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[Windows] Propagate the enabled accessibility state (#184501)" (flutter/flutter#186492) 2026-05-13 srawlins@google.com [dev] Use super parameters in missed spots (flutter/flutter#186193) 2026-05-13 loic.peron@inetum.com [Windows] Propagate the enabled accessibility state (flutter/flutter#184501) 2026-05-13 matt.boetger@gmail.com [flutter_tool] filter out MotionEvent-JNI warning spam from logcat (#174783) (flutter/flutter#186079) 2026-05-13 engine-flutter-autoroll@skia.org Roll Packages from 93cbed6 to 2ec2236 (1 revision) (flutter/flutter#186464) 2026-05-13 mdebbar@google.com [web] Fix untriaged issues link label (flutter/flutter#186465) 2026-05-13 bdero@google.com [Impeller] Namespace user-supplied shaders to prevent entrypoint collisions (flutter/flutter#186332) 2026-05-13 1063596+reidbaker@users.noreply.github.com [flutter_tools] Migrate detectLowCompileSdkVersionOrNdkVersion to AGP task (flutter/flutter#184731) 2026-05-13 jason-simmons@users.noreply.github.com Update the Flutter Gallery web app template files to support running with Wasm (flutter/flutter#186268) 2026-05-13 jason-simmons@users.noreply.github.com [web] Use heap allocation for buffers that would consume too much space on the Wasm stack (flutter/flutter#186228) 2026-05-13 engine-flutter-autoroll@skia.org Roll Skia from 56ca5896c0d9 to 27f7bba22600 (3 revisions) (flutter/flutter#186444) 2026-05-13 engine-flutter-autoroll@skia.org Roll Fuchsia Linux SDK from z7ICmPtn4hspu02zk... to y6uQHA5xUN83IF395... (flutter/flutter#186442) 2026-05-13 engine-flutter-autoroll@skia.org Roll Skia from 6385958d2feb to 56ca5896c0d9 (1 revision) (flutter/flutter#186441) 2026-05-13 engine-flutter-autoroll@skia.org Roll Dart SDK from 9576691c37d8 to 8e30b88e4d5a (1 revision) (flutter/flutter#186429) 2026-05-13 engine-flutter-autoroll@skia.org Roll Skia from 77a21bc723dc to 6385958d2feb (9 revisions) (flutter/flutter#186428) 2026-05-13 164032450+AlexEduV@users.noreply.github.com Docs/improving docs for semantics UI lib (flutter/flutter#186125) 2026-05-12 jason-simmons@users.noreply.github.com [Tool] Support glob patterns when parsing workspaces in FlutterProject (flutter/flutter#185715) 2026-05-12 nico.reiab@gmail.com docs: fix overriden -> overridden in MediaQueryData dartdoc (flutter/flutter#186323) 2026-05-12 brackenavaron@gmail.com [Test cross imports] No material in `test/foundation`, `test/gestures`, `test/semantics`, `test/services` (flutter/flutter#186144) 2026-05-12 nico.reiab@gmail.com docs: fix "tha" -> "that" typo in widget_inspector_test comment (flutter/flutter#186322) 2026-05-12 nico.reiab@gmail.com docs: Fix doubled-word typos in framework dartdoc (flutter/flutter#186319) 2026-05-12 137456488+flutter-pub-roller-bot@users.noreply.github.com Roll pub packages (flutter/flutter#186418) 2026-05-12 30870216+gaaclarke@users.noreply.github.com Bumped required mediatek vender sdk version. (flutter/flutter#186405) 2026-05-12 magder@google.com Make DeepLinkJsonFromManifestTask Gradle task build cacheable (flutter/flutter#185903) 2026-05-12 66727653+ishaq2321@users.noreply.github.com Harden dev tooling scripts against command injection and log leaks (flutter/flutter#186076) 2026-05-12 137456488+flutter-pub-roller-bot@users.noreply.github.com Roll pub packages (flutter/flutter#186274) 2026-05-12 bdero@google.com [Flutter GPU] Allow allocating multi-mip textures and overwriting specific (mip, slice) levels (flutter/flutter#185890) 2026-05-12 zhongliu88889@gmail.com [web] Fix MenuAnchor dismiss when semantics enabled (flutter/flutter#183093) If this roll has caused a breakage, revert this CL and stop the roller using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC bmparr@google.com,stuartmorgan@google.com on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
Resolves #175464
Flutter GPU user shaders, FragmentProgram (RuntimeEffect) shaders, and Impeller's own entity-contents shaders all register into the same per-Context
impeller::ShaderLibrary, keyed only on(entrypoint name, stage). Because impellerc derives entrypoint names from source file stems with no scope prefix, a user-supplied shader whose.fragsource file matches an Impeller-internal one will collide. In the FragmentProgram case the collision is worse than a silent shadow:RuntimeEffectContents::RegisterShaderactively callsUnregisterFunctionon the existing entry when a new dirty stage arrives at the same key (this eviction path was added intentionally to support shader hot reload), so a user could in principle evict an engine pipeline function.This change introduces a small
ShaderKey::MakeUserScopedName(scope, library_id, entrypoint)helper that prefixes user-shader entrypoints with<scope>:<library_id>:before they go into the shared registry. Engine-internal entrypoints stay unprefixed and remain unspoofable: impellerc-generated names are valid identifiers and cannot contain:. AllRegisterFunction/UnregisterFunction/GetFunctionsites in the user-shader paths (and the runtime-effect pipeline cache key inContentContext) are routed through the helper. Engine-internal lookup sites (PipelineBuilder,ComputePipelineBuilder, the runtime-effect vertex stage inRuntimeEffectContents::CreatePipeline) are unchanged because they already use compile-time constant entrypoint names.The
library_idis anchored to a stable per-source identifier so that hot reload semantics are preserved:FragmentProgram::initFromAssetcallsRuntimeStage::SetLibraryId(asset_name). Hot reload of the same.fragasset produces a newRuntimeStagewith the samelibrary_id, so the existing eviction path atRuntimeEffectContents::RegisterShadercontinues to evict and replace at the same scoped key.flutter::gpu::ShaderLibrary::MakeFromAssetplumbs the asset name intoMakeFromFlatbuffer, which propagates it to each ownedflutter::gpu::Shader. A future hot reload story for Flutter GPU can land its own dirty/evict logic against this same scoped key.This also covers the multi-package case for free: Flutter's asset manifest already disambiguates package-shipped assets via the
packages/<pkg>/...prefix, so two packages each shipping a shader with the same internal name land at distinct asset paths and therefore distinctlibrary_idvalues.Tests
ShaderKey::MakeUserScopedName(correct format, contains the:separator that prevents engine-internal collisions, different library ids and different scopes do not collide).RuntimeStage::GetLibraryId(auto-assigned and unique by default, overridable viaSetLibraryId).All added under
impeller/runtime_stage/runtime_stage_unittests.cc.Pre-launch Checklist
///).