Each one of the existing E2E tests now runs AxePuppeteer#analyze() on the relevant portion(s) of the DOM and outputs the returned a11y violations to stderr.

Would it not be reasonable to run the analysis on the entire document after every test case?

Purely from the perspective of the developer experience of authoring tests, I'd neither want this to be possible to overlook, nor a hurdle in making the decision on what should be tested.

Hey Andrew,

Yes, it would absolutely be possible. We opted to only analyze the relevant portions of the DOM for a two major reasons:

1. reduce duplicate issue count
2. reduce the time spent doing analysis

If neither of these were a concern, we would have structured this differently. Rather than editing every test file, we'd just tap into the global afterEach hook and do something like:

afterEach(async () => {
  const axe = new AxePuppeteer(page)
  const results = await axe.analyze()
  logA11yResults(results)
})

This would have been much simpler, and would have found most of the relevant issues, but would have slowed down the test suite by more than a little and would produce a significant number of duplicate issues.

This was superseded by #13241, we still need to integrate new matcher with e2e test suite but this is going to be integrated in the way @aduth suggested:

Would it not be reasonable to run the analysis on the entire document after every test case?

Purely from the perspective of the developer experience of authoring tests, I'd neither want this to be possible to overlook, nor a hurdle in making the decision on what should be tested.

Thanks for all your excellent work done here which helped us to make it solid. Much appreciated. Your involvement a WC US and upfront preparation with axe-puppeteer was amazing ❤️