What is this?

While the current queue system in @percy/core is better than the lazy promise system from @percy/agent, there are still a few performance issues. Namely, multiple browser pages can be opened at the same time for a single snapshot. So while our queue concurrency is set to a default of 5, the 2 widths per snapshot default causes 2 pages to be open per snapshot. Resulting in our true snapshot concurrency being severely limited by the number of widths.

Additionally, when using the capture() method to take DOM snapshots, another browser page is opened to take said snapshot before asset discovery opens pages for all of the widths. While page captures have their own concurrency, this can result in up to 10+ browser pages being opened and closed all at once across the process when capturing more than 5 snapshots.

Finally, while captures and snapshots were managed by their own individual queues, resulting API uploads were not queued. If the capture/snapshot queue drains faster than a few requests can resolve, this could then result in many many in-flight requests which could then clog the process and cause timeouts.

Snapshot, capture, and discovery changes

Rather than open multiple browser pages for multiple widths to discover responsive assets in parallel, we can trigger page resize events sequentially to limit the number of pages that need to be open. This actually results in faster responsive asset discovery for initial resource requests because common requests are no longer duplicated across all widths. While testing, I discovered that in some scenarios the browser will not request smaller assets when a larger one is already loaded. Because of this, widths are sorted with the initial page loading at the smallest width and then resizing to each larger width.

For capturing DOM snapshots with the capture() method, we can eliminate the extraneous browser page by decoupling the snapshot() method and discovering assets simultaneously while capturing the DOM. To achieve this, the discoverer class was broken down into functional methods that can be used by the browser page network when enabling request interception. With this change, the discovery asset cache was moved into the main Percy class and manipulating that cache is now done via request interception resource callbacks.

Now that the capture() method was isolated from the snapshot() method, the logic between them became very similar:

snapshot() {
  // open a page with request interception enabled
  // navigate to a URL and capture assets
  // serve DOM snapshot as root resource
  // schedule snapshot upload
}

capture() {
  // open a page with request interception enabled
  // navigate to a URL and capture assets
  // capture DOM snapshot as root resource
  // schedule snapshot upload
}

Since they were now so similar, these two methods could be merged into a single snapshot() method with one important piece of logic:

// -> if a DOM snapshot was provided, serve it
// -> if no DOM snapshot was provided, capture it

The capture portion of the code remained largely similar to before, but with other asset discovery code now present alongside the capture code, the method became a bit too complex. To alleviate the complexity, capturing of the DOM snapshot was moved into a dedicated page.snapshot() method. The various wait* options were also moved to this method away from the page.goto() method, since those options are only used when capturing a DOM snapshot.

With the percy.snapshot() method now accepting capture options, a snapshot config util was created to validate and merge snapshot options. Part of this validation is ensuring that capture options are not provided along with a static DOM snapshot. The snapshots capture option was also renamed to be additionalSnapshots to avoid confusion with the method name.

Queue changes

Now that captures and snapshots share a method and the discovery process has been broken down, we can combine all three separate queues into a single snapshot queue.

To manage uploads and prevent too many open requests happening at once, an upload queue was also added. The snapshot method will now asynchronously schedule uploads using this queue. This upload queue can also be optionally deferred which allows snapshots to be replaced prior to being sent to Percy. Since uploads can be deferred, it also makes sense to defer build creation along with snapshot uploads. In the future, we can expose this option to allow snapshot retries, or to avoid creating builds when the test suite fails (this would also mean Percy would start at the end of a test run rather than process snapshots as soon as possible).

To facilitate pausing, flushing, removing, and replacing queued tasks, the internal queue class was updated to manage tasks by id. Tasks can also be run according to a priority, which is utilized by the build creation task to ensure it always runs first in the queue. The build creation task also pauses the queue until the build has been created to ensure the build exists before attempting to upload snapshots. Upon build creation failure, the queue is cleared and closed and snapshots will no longer be accepted (the snapshot method will bubble a queue closed error).

Other changes

When the API responds with a 422 build failed error, the queue will also be cleared and closed after logging the error. The @percy/sdk-utils package was updated to catch closed errors and disable future snapshots.

While @percy/client kept build state and mapped one-to-one with a build, build failures are only really tied to closing core queues and preventing future snapshot processing. The client instance does not need to know about this particular state, and other build fetching methods also do not care about state and are not tied to a single build. Therefore it made sense to remove this one-to-one build requirement from client completely. However, this means internal breaking changes to client — now all client methods relating to a build will require a build ID and assertions were added so useful errors are thrown when forgetting to provide one.

With the major parts of core shifting around, other changes were also made to that package. Its src file tree was flattened, util methods were combined, and tests were reorganized to match the new code structure. With both core and client breaking changes, all internal usage amongst the CLI was also updated.

For deferred snapshots, the core stop method was updated to not immediately clear any queued snapshots. It can however, still be told to clear them when the new force argument is true. Caught errors in the CLI are passed along to the command's respective finally() method, which we can leverage to force close the percy instance upon fatal errors.

Various logs were also updated. Redundant start and stop logs, as well as build creation logs were removed; the latter due to potentially being deferred. Logs were also added to client, since we've now long removed Winston from our dependency tree.