Skip to content

feat: Snapshot auto-update workflow for diff grader #85

Description

@spboyer

Summary

Add a --update-snapshots flag to waza run that updates diff grader baseline files when they've drifted, similar to Jest's --updateSnapshot or Go's -update flag.

Motivation

The diff_grader already compares workspace files against expected snapshots. But when intentional changes are made, manually updating all snapshot files is tedious. An auto-update mode captures the current output as the new baseline.

Proposed Implementation

Flag: waza run eval.yaml --update-snapshots

When enabled:

  1. Run tasks normally
  2. For any diff grader that detects differences, instead of failing:
    • Copy the actual output to the expected snapshot path
    • Mark the task as "updated" (not pass/fail)
  3. Report which snapshots were updated

Output

📸 Snapshot Updates:
  ✏️ fixtures/expected/output.md — updated (3 lines changed)
  ✏️ fixtures/expected/config.json — updated (new file)
  ✅ fixtures/expected/readme.txt — no changes

Acceptance Criteria

  • --update-snapshots flag on waza run
  • Diff grader writes actual output to expected path when flag is set
  • Summary of updated vs unchanged snapshots
  • Tests covering: update flow, no-changes flow, new snapshot creation

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestgoPull requests that update go codepriority:p2Next sprintsquad:linusAssigned to Linus (Backend Developer)

Fields

No fields configured for Feature.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions