Skip to content

Fix quality gate to detect per-file analysis stubs and prevent bad output like PR #1463#1465

Merged
pethers merged 4 commits intomainfrom
copilot/analyze-fix-bad-workflow
Mar 30, 2026
Merged

Fix quality gate to detect per-file analysis stubs and prevent bad output like PR #1463#1465
pethers merged 4 commits intomainfrom
copilot/analyze-fix-bad-workflow

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Mar 30, 2026

PR #1463 shipped 10 per-file analyses that were unreplaced script-generated stubs (empty SWOT quadrants, generic boilerplate text, zero Mermaid diagrams). Root cause: the quality gate only glob-checked $ANALYSIS_DIR/*.md — missing everything in documents/. A bash bug (grep -c + || echo 0 producing "0\n0") also broke integer comparisons silently.

Quality gate expansion (SHARED_PROMPT_PATTERNS.md)

  • Check scope: find "$ANALYSIS_DIR" -name "*.md" replaces "$ANALYSIS_DIR"/*.md glob — now covers documents/ subdirectory
  • Check 6 (new): Stub/boilerplate scoring — detects _No strengths identified_, generic perspective text, missing Mermaid diagrams, missing tables. Score ≥ 3 = blocking failure
  • Check 7 (new): Coverage — every .json in documents/ must have a corresponding -analysis.md
  • Anti-pattern warning added to Step 4 listing specific rejected output patterns from pre-article-analysis.ts stubs

Inline quality gate in workflows

  • news-realtime-monitor.md: Replaced vague "see SHARED_PROMPT_PATTERNS" reference with explicit 4-step protocol (A: read templates → B: create per-file analyses → C: rewrite synthesis → D: run quality gate bash block). Step A now references ALL 6 methodology guides + ALL 8 analysis templates via SHARED_PROMPT_PATTERNS Steps 2–3 (not a reduced subset). Gate is blocking before Step 2.
  • news-evening-analysis.md: Added full inline quality gate as step B5 with all 7 checks matching SHARED_PROMPT_PATTERNS (Mermaid diagrams, style directives, [REQUIRED] placeholders, SWOT evidence tables, structured tables, stub detection, and per-file coverage vs JSON count)

Filename convention standardization (all 3 files)

Standardized per-file analysis naming to *-analysis.md (dash convention) throughout — replaced all {id}.analysis.md (dot convention) references in SHARED_PROMPT_PATTERNS, news-realtime-monitor, and news-evening-analysis to match what scripts generate and quality gate checks validate.

Bash bug fix (all 3 files)

# Before — produces "0\n0" when grep finds 0 matches (exit 1 + echo 0)
MERMAID_COUNT=$(grep -c '```mermaid' "$f" 2>/dev/null || echo 0)

# After — grep outputs "0" on no match, || true prevents error propagation without extra output
MERMAID_COUNT=$(grep -c '```mermaid' "$f" 2>/dev/null || true)
if [ "${MERMAID_COUNT:-0}" -eq 0 ]; then  # ${:-0} guards empty string from missing files

Validation

Quality gate now correctly catches all 10 PR #1463 stubs (score 6/6 each). All 55 workflow tests and 68 analysis framework tests pass.

@github-actions github-actions bot added the size-xs Extra small change (< 10 lines) label Mar 30, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🏷️ Automatic Labeling Summary

This PR has been automatically labeled based on the files changed and PR metadata.

Applied Labels: size-xs

Label Categories

  • 🗳️ Content: news, dashboard, visualization, intelligence
  • 💻 Technology: html-css, javascript, workflow, security
  • 📊 Data: cia-data, riksdag-data, data-pipeline, schema
  • 🌍 I18n: i18n, translation, rtl
  • 🔒 ISMS: isms, iso-27001, nist-csf, cis-controls
  • 🏗️ Infrastructure: ci-cd, deployment, performance, monitoring
  • 🔄 Quality: testing, accessibility, documentation, refactor
  • 🤖 AI: agent, skill, agentic-workflow

For more information, see .github/labeler.yml.

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

…te patterns

- SHARED_PROMPT_PATTERNS.md: Expand quality gate to check documents/ subdirectory,
  detect stub patterns (empty SWOT, generic boilerplate), verify coverage
- news-realtime-monitor.md: Add inline quality gate bash block with per-file
  stub detection, add explicit per-file analysis protocol with steps A-D
- news-evening-analysis.md: Add inline quality gate bash block with same checks
- Add anti-pattern warning in SHARED_PROMPT_PATTERNS Step 4 listing specific
  rejected output patterns from script-generated stubs

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/576202f1-0944-4fac-91c7-d3b2974d9a0c

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
@github-actions github-actions bot added documentation Documentation updates workflow GitHub Actions workflows ci-cd CI/CD pipeline changes news News articles and content generation agentic-workflow Agentic workflow changes size-l Large change (250-1000 lines) labels Mar 30, 2026
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

- Replace `$(grep -c ... || echo 0)` with `$(grep -c ... || true)` to prevent
  double "0\n0" output when no matches found
- Add `${VAR:-0}` safety for integer comparisons with potentially empty values
- Use more specific SWOT stub regex: `_No (strengths|weaknesses|...) identified_`
- Verified: quality gate now correctly catches all 10 PR #1463 stubs (score 6/6)

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/576202f1-0944-4fac-91c7-d3b2974d9a0c

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

Copilot AI changed the title [WIP] Analyze and fix agentic workflow issues Fix quality gate to detect per-file analysis stubs and prevent bad output like PR #1463 Mar 30, 2026
Copilot AI requested a review from pethers March 30, 2026 16:03
@pethers pethers marked this pull request as ready for review March 30, 2026 16:08
Copilot AI review requested due to automatic review settings March 30, 2026 16:08
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR strengthens the agentic analysis “quality gate” so it checks all analysis markdown under analysis/daily/$DATE/ (including documents/) and blocks known script-stub/boilerplate per-file analyses—addressing the failure mode seen in PR #1463.

Changes:

  • Expand quality gate scope from "$ANALYSIS_DIR"/*.md to find "$ANALYSIS_DIR" -name "*.md" and add per-file stub scoring + per-file coverage checks in SHARED_PROMPT_PATTERNS.md.
  • Inline a blocking quality gate + more explicit per-file analysis protocol into news-realtime-monitor.md.
  • Add an inline (blocking) quality gate block to news-evening-analysis.md.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 4 comments.

File Description
.github/workflows/news-realtime-monitor.md Adds a blocking per-file analysis protocol and inlined quality gate before proceeding to later workflow steps.
.github/workflows/news-evening-analysis.md Adds an inline, blocking quality gate step to prevent proceeding with stubbed per-file analyses.
.github/workflows/SHARED_PROMPT_PATTERNS.md Updates canonical quality gate to scan documents/, detect stub patterns, and validate per-file analysis coverage.

Comment on lines +395 to +397
1. `view analysis/templates/per-file-political-intelligence.md` — read FULLY, note the required structure
2. `view analysis/methodologies/ai-driven-analysis-guide.md` — read the "BAD vs GOOD" examples
3. `view analysis/methodologies/political-swot-framework.md` — understand evidence tables
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new Step A only instructs reading 3 files, but the repo’s standardized AI-driven analysis protocol requires reading all 6 methodology guides and all 8 analysis templates before writing any analysis. To avoid agents following the shorter list and producing non-compliant output, align this Step A with the SHARED_PROMPT_PATTERNS protocol (or explicitly reference it without reducing the required-reading set).

Suggested change
1. `view analysis/templates/per-file-political-intelligence.md` — read FULLY, note the required structure
2. `view analysis/methodologies/ai-driven-analysis-guide.md` — read the "BAD vs GOOD" examples
3. `view analysis/methodologies/political-swot-framework.md` — understand evidence tables
1. Follow the organization-wide **SHARED_PROMPT_PATTERNS Step A** exactly: read **all 6 methodology guides** and **all 8 analysis templates** defined there (in `analysis/methodologies/` and `analysis/templates/`) **before writing any analysis**. Do NOT subset or skip any required document.
2. After completing SHARED_PROMPT_PATTERNS Step A, (re)read these **news-monitor-specific assets**:
- `view analysis/templates/per-file-political-intelligence.md` — read FULLY, note the required structure
- `view analysis/methodologies/ai-driven-analysis-guide.md` — read the "BAD vs GOOD" examples
- `view analysis/methodologies/political-swot-framework.md` — understand evidence tables

Copilot uses AI. Check for mistakes.
2. For EACH JSON file:
a. Read it with `view` — extract dok_id, titel, datum, parti, organ
b. Apply ALL 6 analytical lenses (classification, SWOT, risk, STRIDE, stakeholders, forward indicators)
c. Write or rewrite `{dok_id}-analysis.md` following the per-file template EXACTLY
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per-file analysis filename convention is inconsistent across the workflow instructions/templates: this section uses {dok_id}-analysis.md, but other parts of the workflow and the per-file template still refer to {id}.analysis.md. Standardize on the actual generated/validated convention (scripts + quality gate expect *-analysis.md) and update the remaining references so agents don’t create files that the coverage/stub checks miss.

Suggested change
c. Write or rewrite `{dok_id}-analysis.md` following the per-file template EXACTLY
c. Write or rewrite the per-file analysis markdown so that its filename matches the `*-analysis.md` convention (for example `{dok_id}-analysis.md`) and follows the per-file template EXACTLY

Copilot uses AI. Check for mistakes.
Comment on lines +146 to 148
3. **Write `{dok_id}-analysis.md`** alongside the data file, following `per-file-political-intelligence.md` template EXACTLY
4. **Include ≥1 Mermaid diagram** with REAL data from the document (not placeholder)
5. **Quality gate**: ≥3 evidence citations with dok_id, confidence labels on all claims, zero `[REQUIRED]` placeholders
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This update switches per-file naming to {dok_id}-analysis.md, but SHARED_PROMPT_PATTERNS still contains other guidance using {id}.analysis.md later in the file and the per-file template currently documents .analysis.md naming. Please make the naming convention consistent across SHARED_PROMPT_PATTERNS and the per-file template so the new coverage/stub checks (which look for *-analysis.md) aren’t undermined by conflicting instructions.

Copilot uses AI. Check for mistakes.
Comment on lines +759 to +813
#### B5. MANDATORY Quality Gate — Run Before Proceeding

> 🚨 **BLOCKING**: Do NOT proceed to article generation or commit until this quality gate passes. If it fails, go back and fix analysis files.

```bash
if [ -z "${ARTICLE_DATE:-}" ]; then
if [ -n "${{ github.event.inputs.article_date }}" ]; then
ARTICLE_DATE="${{ github.event.inputs.article_date }}"
else
ARTICLE_DATE=$(date -u +%Y-%m-%d)
fi
fi
ANALYSIS_DIR="analysis/daily/$ARTICLE_DATE"
QUALITY_PASS=true
FAIL_COUNT=0

echo "=== 🔍 Analysis Quality Gate Check ==="

DAILY_MD_FILES=$(find "$ANALYSIS_DIR" -maxdepth 1 -name "*.md" -type f 2>/dev/null)
PERFILE_MD_FILES=$(find "$ANALYSIS_DIR/documents" -name "*-analysis.md" -type f 2>/dev/null)
ALL_MD_FILES=$(find "$ANALYSIS_DIR" -name "*.md" -type f 2>/dev/null)
echo "📊 Daily: $(echo "$DAILY_MD_FILES" | grep -c '.' 2>/dev/null || true) | Per-file: $(echo "$PERFILE_MD_FILES" | grep -c '.' 2>/dev/null || true)"

# Check 1: Daily synthesis Mermaid diagrams
for f in $DAILY_MD_FILES; do
[ ! -f "$f" ] && continue
if [ "$(grep -c '```mermaid' "$f" 2>/dev/null || true)" -eq 0 ]; then
echo "❌ FAIL: $(basename "$f") has NO Mermaid diagrams"
QUALITY_PASS=false; FAIL_COUNT=$((FAIL_COUNT + 1))
fi
done

# Check 2: No [REQUIRED] placeholders
for f in $ALL_MD_FILES; do
[ ! -f "$f" ] && continue
if [ "$(grep -c '\[REQUIRED\]' "$f" 2>/dev/null || true)" -gt 0 ]; then
echo "❌ FAIL: $(basename "$f") has unfilled [REQUIRED] placeholders"
QUALITY_PASS=false; FAIL_COUNT=$((FAIL_COUNT + 1))
fi
done

# Check 3: Per-file analyses must NOT be stubs
STUB_COUNT=0
for f in $PERFILE_MD_FILES; do
[ ! -f "$f" ] && continue
STUB_SCORE=0
[ "$(grep -cE '_No (strengths|weaknesses|opportunities|threats) identified_' "$f" 2>/dev/null || true)" -ge 2 ] && STUB_SCORE=$((STUB_SCORE + 2))
[ "$(grep -c 'this document requires assessment of\|this document warrants scrutiny for\|this document may affect business' "$f" 2>/dev/null || true)" -ge 2 ] && STUB_SCORE=$((STUB_SCORE + 2))
[ "$(grep -c '```mermaid' "$f" 2>/dev/null || true)" -eq 0 ] && STUB_SCORE=$((STUB_SCORE + 1))
[ "$(grep -c '^|' "$f" 2>/dev/null || true)" -lt 2 ] && STUB_SCORE=$((STUB_SCORE + 1))
if [ "${STUB_SCORE:-0}" -ge 3 ]; then
echo "❌ FAIL: $(basename "$f") is a stub (score=$STUB_SCORE) — MUST be replaced with real analysis"
STUB_COUNT=$((STUB_COUNT + 1)); QUALITY_PASS=false; FAIL_COUNT=$((FAIL_COUNT + 1))
fi
done
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR description says the evening workflow got the “same inline quality gate”, but this block is a simplified subset (e.g., it doesn’t check Mermaid style directives/color-coding, SWOT evidence tables with dok_id, or per-file coverage vs JSON count). Either expand this gate to match the SHARED_PROMPT_PATTERNS checks (including coverage + style directives) or adjust the PR description so expectations match what’s actually enforced.

Copilot uses AI. Check for mistakes.
@pethers
Copy link
Copy Markdown
Member

pethers commented Mar 30, 2026

@copilot apply changes based on the comments in this thread

…lysis.md naming, expand evening quality gate

1. Step A now references ALL 6 methodology guides + ALL 8 templates via
   SHARED_PROMPT_PATTERNS Steps 2-3 (not a reduced 3-file list)
2. Standardized filename convention to *-analysis.md (dash) across all 3 files
   - SHARED_PROMPT_PATTERNS: {id}.analysis.md → {dok_id}-analysis.md
   - news-realtime-monitor: {dok_id}.analysis.md → {dok_id}-analysis.md
   - news-evening-analysis: {id}.analysis.md → {dok_id}-analysis.md
3. Expanded evening-analysis quality gate from 3 checks to 7 checks matching
   SHARED_PROMPT_PATTERNS (added: style directives, SWOT evidence tables,
   structured tables, coverage vs JSON count)

Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/b81b556e-59b9-49c8-b0d2-1744b795d910

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Mar 30, 2026

@copilot apply changes based on the comments in this thread

Applied all 4 review comments in commit a75d934:

  1. Step A expanded — now references ALL 6 methodology guides + ALL 8 templates via SHARED_PROMPT_PATTERNS Steps 2–3, with news-monitor-specific assets listed separately
  2. Filename convention standardized — all {id}.analysis.md (dot) references replaced with {dok_id}-analysis.md (dash) across all 3 files
  3. Evening quality gate expanded — from 3 checks to 7 checks matching SHARED_PROMPT_PATTERNS (added: Mermaid style directives, SWOT evidence tables with dok_id, structured table row counts, per-file coverage vs JSON count)

@github-actions
Copy link
Copy Markdown
Contributor

🔍 Lighthouse Performance Audit

Category Score Status
Performance 85/100 🟡
Accessibility 95/100 🟢
Best Practices 90/100 🟢
SEO 95/100 🟢

📥 Download full Lighthouse report

Budget Compliance: Performance budgets enforced via budget.json

@pethers pethers merged commit ecd6065 into main Mar 30, 2026
13 checks passed
@pethers pethers deleted the copilot/analyze-fix-bad-workflow branch March 30, 2026 16:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agentic-workflow Agentic workflow changes ci-cd CI/CD pipeline changes documentation Documentation updates news News articles and content generation size-l Large change (250-1000 lines) size-xs Extra small change (< 10 lines) workflow GitHub Actions workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants