Driver
PR #340 (closes #325 + #326 + #331 + #332) shipped the count-drift smoke test at .claude/hooks/tests/test_site_counts.sh along with AgDR-0046 documenting the chosen drift-prevention mechanism. AgDR-0046's Consequences section explicitly flagged a v1 risk: pattern coverage is naturally narrow, and any new copy that uses synonyms or wraps a count across a newline is invisible to the smoke test.
Rex's review of #340 (verdict APPROVED, non-blocking nit) found the exact instances of that risk in site/llms-full.txt:
The smoke test missed all three because:
- Pattern phrasing mismatch —
shell +scripts? / mechanical +gates? don't match shell hooks / mechanical enforcement scripts
- Multiline gap — line-by-line grep can't bridge
52\n slash commands (count and noun on separate lines)
This is the exact failure mode AgDR-0046 § Consequences anticipated. Worth closing in v1.1 of the drift detector.
Scope
Single PR, two-part fix:
1. Refresh the 3 stale strings in site/llms-full.txt
- Line 92:
52 → 53 (skills count)
- Line 131:
24 → 29 (hooks count)
- Line 132-133:
52\n slash commands → 53\n slash commands (skills count, multiline)
2. Extend test_site_counts.sh to catch the missed patterns
Add 2 sub-fixes:
a) Broaden the regex patterns to match the synonym phrasings:
- Hooks: add
shell hooks? + enforcement scripts? + mechanical (enforcement )?(scripts?|hooks?) to the hook-count pattern
- Skills: add
(slash )?commands? (already-partial) + ensure [0-9]+\s+slash\s+commands? matches with \s+ (any whitespace including newline)
b) Add a multiline-flatten pre-pass
Before running the line-by-line patterns, also run a "flattened" pass over each file:
# Flatten consecutive whitespace (including newlines) into single spaces
flattened=$(tr -s '[:space:]' ' ' < "$file")
# Then apply the same patterns against the flattened text
This catches 52\n slash commands because after flatten it becomes 52 slash commands which matches [0-9]+ slash commands?.
Report both the line-numbered hits (for actionable fixes) AND the flattened-text hits (for completeness). When flatten-only hits surface, name them as "multiline-spanned drift" so the operator can find them by inspecting the file region rather than expecting line:column precision.
3. Smoke-test the smoke test
Extend .claude/hooks/tests/test_site_counts.sh itself with a self-test: when run with an env flag like APEXYARD_TEST_SMOKE=1, plant deliberate drift in temp fixture files at known synonyms + multiline patterns, run the detector against the fixture, and assert it catches each. Fail loud if any are missed. Keeps the detector honest as patterns evolve.
Acceptance Criteria
Out of scope
- Adopters extending the detector for their own custom count claims (separate feature ticket if requested)
- Auto-fix mode (the detector reports drift; an operator decides whether to refresh manually)
- Cross-file count-aggregation (current scope is per-file)
Glossary
| Term |
Definition |
| Drift detector |
The test_site_counts.sh smoke test that asserts framework counts in site copy match find output |
| Multiline-flatten pre-pass |
Run regex patterns against a whitespace-collapsed version of the file (joins counts and nouns split across newlines) |
| Synonym-coverage gap |
The pattern set missing valid English-language variants of the count phrasing (shell hooks vs shell scripts) |
| AgDR-0046 § Consequences |
Source-of-truth section that flagged exactly this v1 risk |
Refs #340 / Rex review nit (non-blocking) / AgDR-0046 § Consequences
Driver
PR #340 (closes #325 + #326 + #331 + #332) shipped the count-drift smoke test at
.claude/hooks/tests/test_site_counts.shalong with AgDR-0046 documenting the chosen drift-prevention mechanism. AgDR-0046's Consequences section explicitly flagged a v1 risk: pattern coverage is naturally narrow, and any new copy that uses synonyms or wraps a count across a newline is invisible to the smoke test.Rex's review of #340 (verdict APPROVED, non-blocking nit) found the exact instances of that risk in
site/llms-full.txt:52(skills) — should be53post-feat(#299): /mutation-test skill + behaviour-quality sensor #338/refactor(#334): rename /generative-engine-audit → /geo-audit #33924(hooks) and52\n slash commands— should be29and53The smoke test missed all three because:
shell +scripts?/mechanical +gates?don't matchshell hooks/mechanical enforcement scripts52\n slash commands(count and noun on separate lines)This is the exact failure mode AgDR-0046 § Consequences anticipated. Worth closing in v1.1 of the drift detector.
Scope
Single PR, two-part fix:
1. Refresh the 3 stale strings in
site/llms-full.txt52→53(skills count)24→29(hooks count)52\n slash commands→53\n slash commands(skills count, multiline)2. Extend
test_site_counts.shto catch the missed patternsAdd 2 sub-fixes:
a) Broaden the regex patterns to match the synonym phrasings:
shell hooks?+enforcement scripts?+mechanical (enforcement )?(scripts?|hooks?)to the hook-count pattern(slash )?commands?(already-partial) + ensure[0-9]+\s+slash\s+commands?matches with\s+(any whitespace including newline)b) Add a multiline-flatten pre-pass
Before running the line-by-line patterns, also run a "flattened" pass over each file:
This catches
52\n slash commandsbecause after flatten it becomes52 slash commandswhich matches[0-9]+ slash commands?.Report both the line-numbered hits (for actionable fixes) AND the flattened-text hits (for completeness). When flatten-only hits surface, name them as "multiline-spanned drift" so the operator can find them by inspecting the file region rather than expecting line:column precision.
3. Smoke-test the smoke test
Extend
.claude/hooks/tests/test_site_counts.shitself with a self-test: when run with an env flag likeAPEXYARD_TEST_SMOKE=1, plant deliberate drift in temp fixture files at known synonyms + multiline patterns, run the detector against the fixture, and assert it catches each. Fail loud if any are missed. Keeps the detector honest as patterns evolve.Acceptance Criteria
site/llms-full.txtrefreshed (53 skills, 29 hooks)test_site_counts.shpatterns extended with the synonym phrasings from Rex's nit52\n slash commands).claude/hooks/tests/fixtures/test_site_counts/(or similar) with deliberate drift plantedAPEXYARD_TEST_SMOKE=1 bash test_site_counts.shpassessite/*.html(drift detector still PASS green there)Out of scope
Glossary
test_site_counts.shsmoke test that asserts framework counts in site copy matchfindoutputshell hooksvsshell scripts)Refs #340 / Rex review nit (non-blocking) / AgDR-0046 § Consequences