Skip to content

Publish E2E parity and coverage report #3589

@jyaunches

Description

@jyaunches

Summary

Add a visible parity and coverage report to the E2E parity comparison workflow so maintainers can see assertion conversion progress, scenario coverage, and known platform gaps directly from GitHub Actions.

Parent / Epic

Part of #3588.

Scope

  • Render mapped / not-yet-converted / retired assertion counts from test/e2e/docs/parity-map.yaml.
  • Include the scenario coverage matrix from the scenario runner metadata.
  • Surface largest legacy assertion conversion gaps by entrypoint.
  • Call out platform limitations, especially hosted macOS lacking Docker for full onboarding.
  • Publish the report to the workflow step summary and retain it as an artifact.

Current PR

Draft PR: #3587

Acceptance Criteria

  • E2E parity compare workflow publishes a Markdown report to $GITHUB_STEP_SUMMARY.
  • Report artifact is uploaded with other .e2e/parity artifacts.
  • Report uses “assertions not yet converted” language instead of “deferred signal”.
  • Report explicitly identifies macOS full-onboarding limitation due to unavailable Docker.
  • PR lands after the scenario live-execution PR fix(e2e): run scenario workflow for real #3493.

Metadata

Metadata

Assignees

Labels

area: ciCI workflows, checks, release automation, or GitHub Actionsarea: e2eEnd-to-end tests, nightly failures, or validation infrastructurev0.0.53Release target
No fields configured for Enhancement.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions