two TODOs for this:
Some of the test traces/devtoolsLogs are old enough now that they're interfering with code we write by forcing it to handle cases that haven't been relevant for years.
Main issues with easy trace updates:
- they are used in many tests, and any replacements would need to be sure to cover approximately the same test surface area (e.g. if a trace is of a page with redirects that are tested for somewhere, any replacement would also need to have redirects)
- many of them were generated for precisely the purpose of their original PR from bespoke pages with no hint of how to replicate today
To avoid getting back into this situation in a few years, we discussed creating test pages and a method to update test traces. Similar to yarn update:sample-artifacts this will likely still be noisy and fiddly, but it should be much easier to update than trying to parse what was special about a trace from just the name "jumpy-cls-m90.json" and stepping through blame.
There are a lot of test traces, so we can start gradually, e.g. the first TODO above mentions including fallback code for pre-m67 test traces, which is just a handful of them.
The test pages could live in the Lighthouse repo (they could be the smoke tests, even), but we should consider hosting them in https://github.com/ChromeDevTools/performance-stories and sharing them with DevTools perf panel testing.
two TODOs for this:
lighthouse/core/lib/tracehouse/trace-processor.js
Lines 88 to 89 in 1b843de
Some of the test traces/devtoolsLogs are old enough now that they're interfering with code we write by forcing it to handle cases that haven't been relevant for years.
Main issues with easy trace updates:
To avoid getting back into this situation in a few years, we discussed creating test pages and a method to update test traces. Similar to
yarn update:sample-artifactsthis will likely still be noisy and fiddly, but it should be much easier to update than trying to parse what was special about a trace from just the name "jumpy-cls-m90.json" and stepping through blame.There are a lot of test traces, so we can start gradually, e.g. the first TODO above mentions including fallback code for pre-m67 test traces, which is just a handful of them.
The test pages could live in the Lighthouse repo (they could be the smoke tests, even), but we should consider hosting them in https://github.com/ChromeDevTools/performance-stories and sharing them with DevTools perf panel testing.