Skip to content

fix(cli): derive live dogfood fixtures from store#2826

Merged
mergify[bot] merged 2 commits into
mainfrom
fix/live-dogfood-store-fixtures
Jun 7, 2026
Merged

fix(cli): derive live dogfood fixtures from store#2826
mergify[bot] merged 2 commits into
mainfrom
fix/live-dogfood-store-fixtures

Conversation

@tmchow

@tmchow tmchow commented Jun 7, 2026

Copy link
Copy Markdown
Owner

Summary

Live dogfood now primes the printed CLI store with a quiet sync pass when the CLI exposes one, then uses the scoped local SQLite cache to replace ID-shaped example placeholders when companion list calls cannot provide a real ID.

If the store exists but has no matching row, the matrix keeps the synthetic example and marks the row with fixture_source: synthetic, giving retros a reliable way to classify fixture-driven failures.

Closes #2431

Tests

  • go test ./internal/pipeline -run 'TestRunLiveDogfood(StoreFixture|ResolveSuccess|ResolveCommandPositionalsSkipPaths)'
  • go test ./internal/pipeline -run 'TestRunLiveDogfoodStoreFixture'
  • go test ./...

Compound Engineering
Codex

@mergify

mergify Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 require-ready-label-and-ci

Wonderful, this rule succeeded.
  • #changes-requested-reviews-by = 0
  • #review-threads-unresolved = 0
  • check-success = build-and-test
  • check-success = generated-test
  • check-success = go-lint
  • check-success = golden
  • check-success = pr-title
  • check-success = test
  • any of:
    • label = ready-to-merge
    • all of:
      • head = release-please--branches--main
      • title ~= ^chore\(main\): release
  • any of:
    • -files ~= ^(\.github/workflows/|\.github/scripts/|scripts/|\.github/CODEOWNERS$)
    • author = tmchow
    • approved-reviews-by = mvanhorn
    • approved-reviews-by = tmchow
    • author = mvanhorn
  • any of:
    • check-success = Greptile Review
    • label = queued
    • check-neutral = Greptile Review
    • check-skipped = Greptile Review
    • head ~= ^mergify/merge-queue/
    • all of:
      • head = release-please--branches--main
      • title ~= ^chore\(main\): release

@greptile-apps

greptile-apps Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds SQLite-backed fixture resolution to the live dogfood pipeline. When the CLI exposes a sync command, a quiet pre-sync (capped at 5 s) primes the local store, then each companion-failure fallback site queries the store for a real resource ID before falling back to the synthetic example placeholder.

  • Store resolution uses the sqlite3 CLI with a hard 2 s cap; the storeResolved == nPlaceholders guard correctly emits \"store\" only when every placeholder came from the DB.
  • fixtureSource propagation covers happy_path, json_fidelity, and the credential-skip JSON result.
  • Test isolation is achieved by deriving a unique binary name from the test function name, preventing the shared-DB contamination flagged in prior review threads.

Confidence Score: 4/5

Safe to merge for CLIs whose resource type names are plain nouns; the store lookup path has a candidate-generation quirk for path segments ending in 'id' that would misclassify those results as synthetic rather than store-backed.

The three concerns raised in earlier review rounds are addressed: cross-test DB contamination is eliminated by per-test unique binary names, the pre-sync timeout is capped at 5 s, and the mixed store/companion fixture_source tag is handled correctly by the storeResolved counter. The remaining open item is that storeResourceCandidates applies the same id-suffix stripping to CLI path segments as to placeholder names, so a command path like [android, get] would search for resource_type 'andr' rather than 'android' and incorrectly produce synthetic results.

internal/pipeline/live_dogfood.go — storeResourceCandidates and the table-existence check in liveDogfoodStoreFixtureID

Important Files Changed

Filename Overview
internal/pipeline/live_dogfood.go Adds store-backed fixture resolution: pre-sync, SQLite lookup via sqlite3 subprocess, storeResolved counter for accurate fixture_source tagging, and all four companion-failure fallback sites. Logic is sound for typical resource type names; edge case with path segments ending in 'id' can produce wrong SQL candidates.
internal/pipeline/live_dogfood_test.go New tests use per-test unique binary names via liveDogfoodStoreFixtureBinaryName, resolving prior cross-test DB contamination. Mixed store+companion source test and pre-sync timeout test cover key new paths cleanly.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[RunLiveDogfood] --> B[liveDogfoodDefaultDBPath]
    B --> C[runLiveDogfoodPreSync]
    C -->|sync found| D[run sync capped 5s]
    D --> E[Loop over commands]
    E --> F[resolveCommandPositionals]
    F --> G{companion found?}
    G -->|no| K[resolveStoreFixtureID]
    G -->|yes| M[run companion CLI]
    M -->|fail| K
    M -->|ok| N[companion-resolved]
    K -->|ok=true| O[storeResolved++]
    K -->|storeAvail no match| P[return synthetic]
    K -->|no store| Q[skip command]
    O --> R{storeResolved == nPlaceholders?}
    N --> R
    R -->|yes| T[source=store]
    R -->|no| U[source=empty]
    T --> V[set FixtureSource on results]
    U --> V
    P --> W[FixtureSource=synthetic]
Loading

Fix All in Codex Fix All in Claude Code Fix All in Cursor Fix All in Conductor

Reviews (2): Last reviewed commit: "fix(cli): address live dogfood fixture r..." | Re-trigger Greptile

Comment thread internal/pipeline/live_dogfood_test.go
Comment thread internal/pipeline/live_dogfood.go
Comment thread internal/pipeline/live_dogfood.go
@tmchow tmchow added the ready-to-merge Allow Mergify to queue and merge this PR when protections pass label Jun 7, 2026
@tmchow

tmchow commented Jun 7, 2026

Copy link
Copy Markdown
Owner Author

@Mergifyio queue

@mergify mergify Bot added the queued PR is in the Mergify merge queue label Jun 7, 2026
mergify Bot added a commit that referenced this pull request Jun 7, 2026
mergify Bot added a commit that referenced this pull request Jun 7, 2026
mergify Bot added a commit that referenced this pull request Jun 7, 2026
mergify Bot added a commit that referenced this pull request Jun 7, 2026
mergify Bot added a commit that referenced this pull request Jun 7, 2026
@mergify mergify Bot merged commit b2f47e5 into main Jun 7, 2026
31 checks passed
@mergify mergify Bot deleted the fix/live-dogfood-store-fixtures branch June 7, 2026 23:36
@mergify

mergify Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

Merge Queue Status

This pull request spent 23 minutes 57 seconds in the queue, including 23 minutes 38 seconds running CI.

Required conditions to merge

@mergify mergify Bot removed the queued PR is in the Mergify merge queue label Jun 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-merge Allow Mergify to queue and merge this PR when protections pass

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[scorer] Dogfood matrix should derive parent-ID fixtures from the synced local store

1 participant