The -G and -A flags can be used to run the gathering and auditing phases separately in the CLI. We should add some way to run these phases separately in user flows.

Previous discussion: #11623

Benefits of 2-stage User Flows

• Faster audit workflow: User flows can take arbitrarily long. Would be nice to test audit changes in flows using existing artifacts without running an entire user flow.
• Deterministic sample updating: (TL;DR sample_v2.json for user flows) To rebaseline the sample flow report, we currently run an end-to-end user flow experience. This means normal page variance tends to bloat the size of the sample diff, making reviews of sample diffs hard and CI verification impossible. Separate gathering modes allow us to check-in an artifacts file and test how changes to the auditing phase affect the final flow result deterministically.
• Less stalling in the middle of a flow: We could defer auditing for each step to the end of the entire flow. Small impact compared to running gatherers, but should come for free with these changes.

Implementation Plan

• Create helper to handle gather phase core(runner): asset manager helper #13519

  • gatherArtifactsOrLoadFromFile() or something
    • Thinking about making this a static function on Runner but open to suggestions here.
  • We won't handle the -G logic for user flows, but it's important that individual navigations respond to the -G flag to maintain parity when running Lighthouse CLI with the --fraggle-rock flag.

• Remove gathering phase from Runner.run() core(runner): independent gather and audit functions #13569

  • Current interface:

    lighthouse/lighthouse-core/runner.js

    Lines 30 to 36 in 68ba77a

    	/**
    	* @template {LH.Config.Config | LH.Config.FRConfig} TConfig
    	* @param {(runnerData: {requestedUrl: string, config: TConfig, driverMock?: Driver}) => Promise<LH.Artifacts>} gatherFn
    	* @param {{config: TConfig, computedCache: Map<string, ArbitraryEqualityMap>, url?: string, driverMock?: Driver}} runOpts
    	* @return {Promise<LH.RunnerResult|undefined>}
    	*/
    	static async run(gatherFn, runOpts) {

  • Proposed interface:

      /**
       * @template {LH.Config.Config | LH.Config.FRConfig} TConfig
       * @param {LH.Artifacts} artifacts
       * @param {{config: TConfig, ...}} runOpts
       * @return {Promise<LH.RunnerResult|undefined>}
       */
      static async run(artifacts, runOpts) {

  • Effectively turns Runner.run() into and "audit runner".

  • Call to function from step 1 lifted to lighthouse-core/run.js and navigation runner all FR runners. Results are passed to Runner.run().

  • Timespan and snapshot runners would directly run whatever is in their gatherFn and pass results as a param to Runner.

  • RFC: core(runner): move non-audit code to index and gather-runner #12393

• Move FR usage of Runner.run to user-flow.js and api.js core(fr): separate audit phase for flows #13623
  - Individual FR runners snapshot, startTimespan, navigation will return artifacts instead of a RunnerResult.
  - Add wrapper functions that send returned artifacts through Runner.run() in api.js. Maintains the current API for individual runners.
  - Artifacts will be saved on an instance of UserFlow. Auditing can be run as part of UserFlow.getFlowResult.

• Add saveArtifacts and loadArtifacts functions to UserFlow
  - Pass artifacts into the constructor?

• Update sample flow JSON script runs audit mode on some checked-in artifacts.

• Add CI check for sample flow JSON