Phase 11: Final Audit Reconciliation

Parent epic: #3588

Goal

Final reconciliation of the audit-coverage work. Every audit row must be evidence-complete or audit-amended with reviewer-visible rationale; every placeholder/no-op assertion group must be removed, implemented, or independently amended out of scope; deterministic final reports must reconcile scenario contracts, expected states, suite steps, assertion modules, and evidence reports without deleting legacy executable E2E tests.

Audit rows in scope

Reconciliation tasks

• Remove, replace, or independently amend every placeholder assertion group that would otherwise hide an audit row
• Reconcile docs/e2e-audit-work-queue.md against final scenario contracts, expected states, suite steps, assertion modules, and evidence reports
• Update minimal test/e2e-scenario/docs/README.md and MIGRATION.md sections needed to explain how to run the migrated coverage and read the audit-row report
• Resolve TODOs introduced by this implementation
• Generate deterministic final coverage + audit-row report
• Do not delete legacy executable E2E tests in this phase — list deletion candidates only

Required deliverables

Reports

• test/e2e-scenario/runtime/reports/final-audit-row-report.json (machine-readable)
• test/e2e-scenario/runtime/reports/final-audit-row-report.md (human-readable)
• Reports byte-identical across two consecutive runs (deterministic)

Docs

• test/e2e-scenario/docs/README.md updated with: contract model, status vocabulary, final wave status
• test/e2e-scenario/docs/MIGRATION.md updated with: current migration state, deletion-readiness criteria

Hygiene tests

• Vitest test that fails if any queue row is marked complete with metadata-only evidence
• Vitest test that fails if any unresolved row lacks an owner / follow-up
• Vitest test that fails if any placeholder/no-op assertion group remains marked complete
• Vitest test that fails if any legacy test/e2e/test-*.sh is deleted in this phase

Validation scenarios — all must pass in PR workflow artifacts

Scenario 11.1 — Final work queue has no completed metadata-only or ownerless unresolved entries (Happy Path)

• Given docs/e2e-audit-work-queue.md and final scenario evidence reports available.
• When Final hygiene validation runs.
• Then No metadata-only row is presented as complete audit coverage; every unresolved row has owner/follow-up; completed rows have executable evidence or independent audit amendment rationale.
• Steps
  1. Read queue rows AQ-001…AQ-065; collect final PR workflow evidence reports.
  2. Run final hygiene tests in workflow.
  3. Verify every queue row is present in PR workflow evidence with evidence-complete or approved audit-amended; no completed metadata-only rows; no ownerless unresolved rows remain.

Scenario 11.2 — Placeholder assertion groups are removed, implemented, or independently amended out of scope (Sad Path)

• Given A placeholder/no-op assertion group remains and is marked complete.
• When Final hygiene validation runs.
• Then Validation fails until the placeholder is removed, replaced by a real module, or covered by an independent audit amendment rationale.
• Steps
  1. Filter rows AQ-063…AQ-065; leave or inject placeholder assertion group in final metadata.
  2. Run final hygiene tests.
  3. Verify failure; placeholder group named; affected queue rows unresolved until real module or approved amendment exists.

Scenario 11.3 — Docs and deterministic reports are updated without deleting legacy scripts (Happy Path)

• Given Implementation is complete and legacy scripts remain in place.
• When Docs and reports are checked.
• Then test/e2e-scenario/docs/README.md and test/e2e-scenario/docs/MIGRATION.md document contract model/status vocabulary/final wave status; generated reports are deterministic; legacy script deletion candidates are listed but scripts are not deleted.
• Steps
  1. Filter rows AQ-063…AQ-065; render reports twice; list legacy test/e2e/test-*.sh scripts; collect PR workflow artifacts.
  2. Compare report outputs and run docs hygiene tests.
  3. Verify rows evidence-complete only when artifacts prove docs contain required sections, reports are byte-identical, legacy scripts still exist.

Acceptance criteria — issue is NOT DONE until ALL are true

1. PR landed in test/e2e-scenario/ with reports, docs, and hygiene tests above.
2. PR CI passing:
   • All hygiene tests pass
   • Report deterministic (byte-identical across two runs)
   • No queue row marked evidence-complete without stable assertion ID + evidence path
   • No unresolved row without owner / follow-up
3. Validation Scenarios 11.1–11.3 all pass in PR workflow artifacts.
4. Audit work queue final pass: every row AQ-001 through AQ-062 is evidence-complete or audit-amended with reviewer-visible rationale; AQ-063/064/065 flipped to evidence-complete.
5. Phase-specific validation gate (from spec): no metadata-only entry presented as satisfying an audit row; no unresolved audit row remains without owner / follow-up; all fixture cleanup/restore obligations documented and tested; legacy executable E2E tests remain in place — deletion is outside this spec.
6. No-cheat gate: zero metadata-only completions.
7. Legacy preservation gate: every legacy test/e2e/test-*.sh from the spec's in-scope list still exists in the repo.
8. PR description references AQ-063/064/065, links this issue.

Dependencies

• All of Phases 1–10 must be evidence-complete before this phase can complete.
• Audit work queue rows AQ-001 through AQ-062 fully resolved.

Out of scope

• Legacy script deletion (handled as a separate later approval, outside this scope)
• New scenario coverage (must be added in earlier phases or as audit amendment)

Cross-phase acceptance gates (apply to every phase)

1. Setup gate — scenario contract declares environment, manifest or no-manifest reason, fixtures, runtime actions, assertions.
2. No-cheat gate — preview/dry-run output cannot mark an audit row complete.
3. Boundary gate — assertions touch the same SUT boundary as the legacy script.
4. Evidence gate — every assertion emits an evidence path and stable assertion ID.
5. Secret gate — no manifest, log, report, or fixture file contains raw secrets.
6. Cleanup gate — fixtures that mutate host or repo state have restore/cleanup logic and tests.
7. Audit completeness gate — every assigned audit row has owner, planned scenario/assertion, phase assignment, evidence status.
8. Phase completion gate — phase complete only when every assigned row has executable evidence (or independent audit amendment).
9. Executable assertion gate — completed scenarios point to concrete suite steps / assertion modules, not pendingStep(...), TODOs, generic probes, or prose.