Re-prioritize pipeline compile jobs and perform them eagerly instead of waiting. #180022

chinmaygarde · 2025-12-17T21:36:51Z

Also, removes "lazy shader mode" and re-enables eager PSO pre-flight.

Impeller eagerly sets up the entire set of pipeline state objects (PSOs) it may need during renderer setup. This works fine on mid to high-end devices as the setup completes well before the first frame is rendered.

However on low-end devices, not all PSOs may be ready before the first frame is rendered. Low-end device usually don't have as much available concurrency and are slower to boot. On these devices, the rendering thread had to wait for the background compile job to be completed. It was also observed that the relatively higher priority render thread was waiting on a background thread to finish a PSO compile job. The PSO compile job could also be stuck behind another job that was not immediately needed. This made the situation on low end devices less than ideal.

To ameliorate this situation, a stop-gap was introduced called "lazy shader mode". This disabled PSO pre-flight entirely and only dispatched jobs when they were needed. This ameliorated somewhat the issue of unneeded jobs blocking the eagerly needed job but the other issues remained. It also meant that one could expect jank the first time a new PSO was needed. This mode was not widely used or advertised.

In this patch, the lazy shader mode has been entirely removed and eager PSO pre-flight is the default in all configurations. Pipeline compile jobs are now handled by a separate compile queue. This queue operates like a transparent concurrent worker pool job dispatcher in the usual case. However, when the pipeline for a specific descriptor is needed, instead of waiting for the job to complete, the queue allows the job for that descriptor to skip to the head of the queue and be performed on the calling thread. This effectively make the calling thread do the job instead of waiting. Another nice side effect of this behavior is that the higher priority thread is now doing the job it was previously waiting on.

This fixes all issues on lower end devices except one. During eager pre-flight, it would still be better if PSO compile jobs for that were somehow divined to be needed first were towards the front of the pre-flight queue. After all, only a handful of rendering features account for the majority of rendering operations. A future augmentation to the compile queue would be to allow higher prioritization of more widely used pipeline compile jobs as instrumented in #176660. This task is tracked in #176665.

This patch only wires up the compile queue for Vulkan. #176657 tracks doing the same for OpenGL.

The efficacy of the re-prioritizations can be traced using the
PrioritiesElevated trace counter. On higher end devices, there should
be zero of these.

Fixes #176656
Fixes #176663
Fixes #176658

flutter-dashboard · 2025-12-17T21:36:55Z

It looks like this pull request may not have tests. Please make sure to add tests or get an explicit test exemption before merging.

If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix?

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing. If you believe this PR qualifies for a test exemption, contact "@test-exemption-reviewer" in the #hackers channel in Discord (don't just cc them here, they won't see it!). The test exemption team is a small volunteer group, so all reviewers should feel empowered to ask for tests, without delegating that responsibility entirely to the test exemption group.

gemini-code-assist

Code Review

This pull request introduces a PipelineCompileQueue to re-prioritize pipeline compilation jobs, allowing them to be performed eagerly instead of waiting. This change removes the old "lazy shader mode" and aims to improve performance on low-end devices by reducing jank caused by waiting for Pipeline State Objects (PSOs). The implementation is currently wired up for Vulkan. The overall approach is sound and the new queue is well-encapsulated. I've provided a few suggestions to improve the implementation of the new PipelineCompileQueue class, focusing on C++ best practices and efficiency.

gemini-code-assist · 2025-12-17T21:40:00Z

engine/src/flutter/impeller/renderer/pipeline_compile_queue.cc

+std::shared_ptr<PipelineCompileQueue> PipelineCompileQueue::Create(
+    std::shared_ptr<fml::ConcurrentTaskRunner> worker_task_runner) {
+  return std::shared_ptr<PipelineCompileQueue>(
+      new PipelineCompileQueue(std::move(worker_task_runner)));
+}


The Google C++ Style Guide, which this repository follows, recommends using std::make_shared instead of std::shared_ptr(new ...) for better performance and exception safety. std::make_shared performs a single memory allocation for both the object and its control block.

To adopt this, you can make the PipelineCompileQueue constructor public. The static Create method will still serve as the designated factory function.

return std::make_shared<PipelineCompileQueue>(std::move(worker_task_runner));

References

C++ code should follow the Google C++ Style Guide. ^(link)

Can't do gem gem. That method is private.

And the suggestion of making the ctor public is a no-go because std::enable_shared_from_this cannot be enforced if someone decides to create a non-shared-pointer owned instance of the queue. The weak_from_this will crash.

gemini-code-assist · 2025-12-17T21:40:00Z

engine/src/flutter/impeller/renderer/pipeline_compile_queue.cc

+  if (!insertion_result.second) {
+    // This bit is being extremely conservative. If insertion did not take
+    // place, someone gave the compile queue a job for the same description.
+    // This is highly unusual but technically not impossible. Just run the job
+    // eagerly.
+    FML_LOG(ERROR) << "Got multiple compile jobs for the same descriptor. "
+                      "Running eagerly.";
+    // Don't invoke the job here has there are we have currently acquired a
+    // mutex.
+    worker_task_runner_->PostTask(job);
+    return true;
+  }


The current logic for handling a duplicate job descriptor logs an error and then posts the job again. This could lead to redundant work if the original job is already in the queue. Since PipelineLibraryVK should prevent duplicate job submissions for the same descriptor, this case is rare. However, if it does occur, it would be safer and more efficient to simply ignore the duplicate request. The original job will eventually complete, and the caller will receive the result via the shared future.

if (!insertion_result.second) { // This bit is being extremely conservative. If insertion did not take // place, someone gave the compile queue a job for the same description. // This is highly unusual but technically not impossible. The job is // already in the queue, so we can ignore this request. return true; }

then posts the job again

Yes, but not to the queue.

it would be safer and more efficient to simply ignore the duplicate request

More efficient yes. But safer, not really. What if the two different futures are awaited upon for the same descriptor with different jobs? As mentioned, this is a rare edge case. But this will limit the damage (and log) in such cases instead of causing a deadlock.

gemini-code-assist · 2025-12-17T21:40:00Z

engine/src/flutter/impeller/renderer/pipeline_compile_queue.cc

+void PipelineCompileQueue::FinishAllJobs() {
+  // This doesn't have to be fast. Just ensures the task queue is flushed when
+  // the compile queue is shutting down with jobs still in it.
+  while (true) {
+    bool has_jobs = false;
+    {
+      Lock lock(pending_jobs_mutex_);
+      has_jobs = !pending_jobs_.empty();
+    }
+    if (!has_jobs) {
+      return;
+    }
+    DoOneJob();
+  }
+}


The current implementation of FinishAllJobs can be inefficient as it repeatedly locks and unlocks the mutex for each job in the queue. A more efficient approach would be to lock the mutex once, move all pending jobs to a local container using std::swap, and then execute them after releasing the lock. This minimizes lock contention, especially if there are many jobs to process during shutdown.

void PipelineCompileQueue::FinishAllJobs() { // This doesn't have to be fast. Just ensures the task queue is flushed when // the compile queue is shutting down with jobs still in it. decltype(pending_jobs_) jobs_to_run; { Lock lock(pending_jobs_mutex_); jobs_to_run.swap(pending_jobs_); } for (const auto& job : jobs_to_run) { job.second(); } }

My reasoning for doing it this way was that any background task runners still running would still be able to participate in job selection. I agree that from the perspective of holding onto the mutex, doing it one shot is better.

Add a comment noting that worker threads may still be running jobs during and after the execution of FinishAllJobs.

jason-simmons

Overall LGTM

jason-simmons · 2025-12-17T23:21:01Z

engine/src/flutter/impeller/renderer/pipeline_compile_queue.cc

+    return false;
+  }
+
+  Lock lock(pending_jobs_mutex_);


Reduce the section that holds the lock to just the pending_jobs_.insert call

jason-simmons · 2025-12-17T23:23:40Z

engine/src/flutter/impeller/renderer/pipeline_compile_queue.cc

+  if (pending_jobs_.empty()) {
+    return nullptr;
+  }
+  auto job_iterator = pending_jobs_.begin();


Should PipelineCompileQueue track the insertion order of the jobs?

The Queue naming and the mention of prioritization imply that jobs will be executed in the order they were posted. But TakeNextJob obtains the job from a std::unordered_map iterator that will select an arbitrary job.

My sense was that it doesn't matter. At least not till we implement #176665 which I have left as a follow up task.

jason-simmons · 2025-12-17T23:26:56Z

engine/src/flutter/impeller/renderer/pipeline_compile_queue.cc

+void PipelineCompileQueue::FinishAllJobs() {
+  // This doesn't have to be fast. Just ensures the task queue is flushed when
+  // the compile queue is shutting down with jobs still in it.
+  while (true) {
+    bool has_jobs = false;
+    {
+      Lock lock(pending_jobs_mutex_);
+      has_jobs = !pending_jobs_.empty();
+    }
+    if (!has_jobs) {
+      return;
+    }
+    DoOneJob();
+  }
+}


Add a comment noting that worker threads may still be running jobs during and after the execution of FinishAllJobs.

zanderso · 2025-12-19T21:20:55Z

Looks like some Impeller/Vulkan unit tests are unhappy with the changes to pipeline lifetime?

…of waiting. Also, removes "lazy shader mode" and re-enables eager PSO pre-flight. Impeller eagerly sets up the entire set of pipeline state objects (PSOs) it may need during renderer setup. This works fine on mid to high-end devices as the setup completes well before the first frame is rendered. However on low-end devices, not all PSOs may be ready before the first frame is rendered. Low-end device usually don't have as much available concurrency and are slower to boot. On these devices, the rendering thread had to wait for the background compile job to be completed. It was also observed that the relatively higher priority render thread was waiting on a background thread to finish a PSO compile job. The PSO compile job could also be stuck behind another job that was not immediately needed. This made the situation on low end devices less than ideal. To ameliorate this situation, a stop-gap was introduced called "lazy shader mode". This disabled PSO pre-flight entirely and only dispatched jobs when they were needed. This ameliorated somewhat the issue of unneeded jobs blocking the eagerly needed job but the other issues remained. It also meant that one could expect jank the first time a new PSO was needed. This mode was not widely used or advertised. In this patch, the lazy shader mode has been entirely removed and eager PSO pre-flight is the default in all configurations. Pipeline compile jobs are now handled by a separate compile queue. This queue operates like a transparent concurrent worker pool job dispatcher in the usual case. However, when the pipeline for a specific descriptor is needed, instead of waiting for the job to complete, the queue allows the job for that descriptor to skip to the head of the queue and be performed on the calling thread. This effectively make the calling thread do the job instead of waiting. Another nice side effect of this behavior is that the higher priority thread is now doing the job it was previously waiting on. This fixes all issues on lower end devices except one. During eager pre-flight, it would still be better if PSO compile jobs for that were somehow divined to be needed first were towards the front of the pre-flight queue. After all, only a handful of rendering features account for the majority of rendering operations. A future augmentation to the compile queue would be to allow higher prioritization of more widely used pipeline compile jobs as instrumented in flutter#176660. This task is tracked in flutter#176665. This patch only wires up the compile queue for Vulkan. flutter#176657 tracks doing the same for OpenGL. Fixes flutter#176656 Fixes flutter#176663 Fixes flutter#176658

chinmaygarde · 2026-01-09T21:19:20Z

Okie dokie. All tests are passing now. The failures were due to the extra logging. When we tore down the renderer without all compile jobs finishing, the aborted jobs would log an error. Before this patch, that was an error because shutdown could only cleanly happen after a clean startup, and a clean startup depended on all compile jobs finishing. This is no longer the case. The upside is also that tests that startup and shutdown the renderer quickly should not finish faster.

chinmaygarde · 2026-01-09T21:47:45Z

The one Google testing failure doesn't seem related. Some sort of a bluetooth scan failing with a timeout. I think this is good to go.

…instead of waiting. (flutter/flutter#180022)

Roll Flutter from 01d37bc to 73769a2 (65 revisions) flutter/flutter@01d37bc...73769a2 2026-01-10 engine-flutter-autoroll@skia.org Roll Dart SDK from 5e855c2bb3ef to 87fbfd5381b6 (1 revision) (flutter/flutter#180800) 2026-01-10 engine-flutter-autoroll@skia.org Roll Skia from b2b109f0e980 to f39cc645b1dd (2 revisions) (flutter/flutter#180796) 2026-01-10 engine-flutter-autoroll@skia.org Roll Dart SDK from b7963905e6a2 to 5e855c2bb3ef (2 revisions) (flutter/flutter#180794) 2026-01-10 ahmedsameha1@gmail.com Make sure that a CupertinoTabScaffold doesn't crash in 0x0 environment (flutter/flutter#179824) 2026-01-10 engine-flutter-autoroll@skia.org Roll Dart SDK from d25ad331b7ea to b7963905e6a2 (2 revisions) (flutter/flutter#180783) 2026-01-10 ahmedsameha1@gmail.com Make sure that a Container doesn't crash in 0x0 environment (flutter/flutter#180350) 2026-01-10 ahmedsameha1@gmail.com Make sure that an Expansible doesn't crash in 0x0 environment (flutter/flutter#180478) 2026-01-10 98614782+auto-submit[bot]@users.noreply.github.com Reverts "Enabled some disabled impeller fragment shader dart tests (#180759)" (flutter/flutter#180785) 2026-01-09 dkwingsmt@users.noreply.github.com Merge `widget_tester_leaks_free_test.dart` into `widget_tester_test.dart` (flutter/flutter#180600) 2026-01-09 codefu@google.com fix: there are no riscv fuchsia artifacts (flutter/flutter#180779) 2026-01-09 engine-flutter-autoroll@skia.org Roll Skia from 7386219151e6 to b2b109f0e980 (1 revision) (flutter/flutter#180771) 2026-01-09 chinmaygarde@google.com Re-prioritize pipeline compile jobs and perform them eagerly instead of waiting. (flutter/flutter#180022) 2026-01-09 30870216+gaaclarke@users.noreply.github.com Enabled some disabled impeller fragment shader dart tests (flutter/flutter#180759) 2026-01-09 engine-flutter-autoroll@skia.org Roll Fuchsia Linux SDK from rxeg-6UB678HKJ4UQ... to 83Favz_zzMzdVuOHg... (flutter/flutter#180765) 2026-01-09 jhy03261997@gmail.com [A11y ] Add `clearSemantics`in table (flutter/flutter#180665) 2026-01-09 chinmaygarde@google.com Update CODEOWNERS to remove chinmaygarde. (flutter/flutter#180703) 2026-01-09 vhaudiquet343@hotmail.fr [ Tool ] Add support for linux riscv64 architecture (flutter/flutter#178711) 2026-01-09 engine-flutter-autoroll@skia.org Roll Skia from e9b3264ade0c to 7386219151e6 (12 revisions) (flutter/flutter#180754) 2026-01-09 engine-flutter-autoroll@skia.org Roll Dart SDK from fe2ba2c5dd50 to d25ad331b7ea (10 revisions) (flutter/flutter#180741) 2026-01-09 98614782+auto-submit[bot]@users.noreply.github.com Reverts "Unpin google_mobile_ads (#180573)" (flutter/flutter#180761) 2026-01-09 koichi20021217@gmail.com Fix typo in dropdown_menu.dart (flutter/flutter#180172) 2026-01-09 engine-flutter-autoroll@skia.org Roll Packages from 039a026 to 51fe1d9 (1 revision) (flutter/flutter#180742) 2026-01-09 goderbauer@google.com Unpin google_mobile_ads (flutter/flutter#180573) 2026-01-09 evanwall@buffalo.edu Update flutter changelog for 3.38.6 (flutter/flutter#180708) 2026-01-08 98614782+auto-submit[bot]@users.noreply.github.com Reverts "Fix iOS xattr removal to clear all extended attributes (#180355)" (flutter/flutter#180709) 2026-01-08 ahmedsameha1@gmail.com Make sure that an EditableText doesn't crash in 0x0 environment (flutter/flutter#180457) 2026-01-08 matt.kosarek@canonical.com Implementation of tooltip windows for win32 (flutter/flutter#179147) 2026-01-08 mdebbar@google.com [web] Don't serve files outside of project (flutter/flutter#180699) 2026-01-08 engine-flutter-autoroll@skia.org Roll Skia from 837be28dd218 to e9b3264ade0c (1 revision) (flutter/flutter#180702) 2026-01-08 sokolovskyi.konstantin@gmail.com Add new motion accessibility features to iOS. (flutter/flutter#178102) 2026-01-08 104009581+Saqib198@users.noreply.github.com Fix iOS xattr removal to clear all extended attributes (flutter/flutter#180355) 2026-01-08 bkonyi@google.com [ Widget Preview ] Move widget_preview_scaffold tests to `dev/integration_tests/widget_preview_scaffold` (flutter/flutter#180658) 2026-01-08 30870216+gaaclarke@users.noreply.github.com De-interleaves engine dart test output (flutter/flutter#180651) 2026-01-08 zhongliu88889@gmail.com [web] Fix SemanticsService.announce not working inside dialogs (flutter/flutter#179958) 2026-01-08 engine-flutter-autoroll@skia.org Roll Skia from 42233226ac56 to 837be28dd218 (2 revisions) (flutter/flutter#180693) 2026-01-08 iinozemtsev@google.com Roll Dart SDK to 3.11.0-296.2.beta (flutter/flutter#180685) 2026-01-08 116356835+AbdeMohlbi@users.noreply.github.com Improve code quality in `AndroidTouchProcessorTest.java` (flutter/flutter#180583) 2026-01-08 1961493+harryterkelsen@users.noreply.github.com Revert "Reverts "[reland] Unify canvas creation and Surface code in S…kwasm and CanvasKit (#179473)" (#180152)" (flutter/flutter#180610) 2026-01-08 engine-flutter-autoroll@skia.org Roll Skia from a0c407bce408 to 42233226ac56 (4 revisions) (flutter/flutter#180688) 2026-01-08 bkonyi@google.com [ Tool ] Fix flake in overall_experience_test.dart (flutter/flutter#180655) 2026-01-08 engine-flutter-autoroll@skia.org Roll Packages from 9705815 to 039a026 (6 revisions) (flutter/flutter#180684) 2026-01-08 104147021+MohammedTarigg@users.noreply.github.com flutter_tools: Auto-generate ExportOptions.plist for manual iOS code signing (flutter/flutter#177888) 2026-01-08 engine-flutter-autoroll@skia.org Roll Skia from 58837e160874 to a0c407bce408 (2 revisions) (flutter/flutter#180679) 2026-01-08 engine-flutter-autoroll@skia.org Roll Skia from 1e3266fdba86 to 58837e160874 (1 revision) (flutter/flutter#180677) 2026-01-08 engine-flutter-autoroll@skia.org Roll Skia from 3c47ea10638f to 1e3266fdba86 (4 revisions) (flutter/flutter#180675) 2026-01-08 engine-flutter-autoroll@skia.org Roll Fuchsia Linux SDK from dTvN_JVSCfGFRasvH... to rxeg-6UB678HKJ4UQ... (flutter/flutter#180673) ...

chinmaygarde requested review from jason-simmons and walley892 December 17, 2025 21:36

github-actions bot added engine flutter/engine related. See also e: labels. e: impeller Impeller rendering backend issues and features requests labels Dec 17, 2025

gemini-code-assist bot reviewed Dec 17, 2025

View reviewed changes

chinmaygarde force-pushed the parco branch from 95066f1 to d8e01b1 Compare December 17, 2025 21:41

jason-simmons reviewed Dec 17, 2025

View reviewed changes

chinmaygarde force-pushed the parco branch 2 times, most recently from e72f115 to 335b78a Compare December 18, 2025 22:41

chinmaygarde requested a review from a team as a code owner December 18, 2025 22:41

github-actions bot added platform-android Android applications specifically team-android Owned by Android platform team labels Dec 18, 2025

jason-simmons approved these changes Dec 18, 2025

View reviewed changes

chinmaygarde force-pushed the parco branch from 335b78a to e52bf53 Compare January 9, 2026 19:14

chinmaygarde added the autosubmit Merge PR when tree becomes green via auto submit App label Jan 9, 2026

auto-submit bot added this pull request to the merge queue Jan 9, 2026

Merged via the queue into flutter:master with commit 07485a2 Jan 9, 2026
181 checks passed

flutter-dashboard bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Jan 9, 2026

engine-flutter-autoroll mentioned this pull request Jan 10, 2026

Roll Flutter from 01d37bc80fd5 to 0b7596b3f269 (64 revisions) flutter/packages#10768

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jan 10, 2026

07485a2 Re-prioritize pipeline compile jobs and perform them eagerly …

a5510f4

…instead of waiting. (flutter/flutter#180022)

engine-flutter-autoroll mentioned this pull request Jan 10, 2026

Roll Flutter from 01d37bc80fd5 to 73769a220639 (65 revisions) flutter/packages#10769

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jan 10, 2026

07485a2 Re-prioritize pipeline compile jobs and perform them eagerly …

8c937cf

…instead of waiting. (flutter/flutter#180022)

engine-flutter-autoroll mentioned this pull request Jan 10, 2026

Roll Flutter from 01d37bc80fd5 to 73769a220639 (65 revisions) flutter/packages#10770

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jan 10, 2026

07485a2 Re-prioritize pipeline compile jobs and perform them eagerly …

3b592b3

…instead of waiting. (flutter/flutter#180022)

Re-prioritize pipeline compile jobs and perform them eagerly instead of waiting. #180022

Re-prioritize pipeline compile jobs and perform them eagerly instead of waiting. #180022

Conversation

chinmaygarde commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

flutter-dashboard bot commented Dec 17, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

chinmaygarde Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist bot Dec 17, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jason-simmons left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zanderso commented Dec 19, 2025

Uh oh!

chinmaygarde commented Jan 9, 2026

Uh oh!

chinmaygarde commented Jan 9, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

chinmaygarde commented Dec 17, 2025 •

edited

Loading

chinmaygarde Dec 17, 2025 •

edited

Loading