Address review comments: expand Step A reading list, standardize -analysis.md naming, expand evening quality gate

Copilot · pethers · Copilot · commit a75d934948cc · 2026-03-30T16:22:24.000Z
1. Step A now references ALL 6 methodology guides + ALL 8 templates via SHARED_PROMPT_PATTERNS Steps 2-3 (not a reduced 3-file list) 2. Standardized filename convention to *-analysis.md (dash) across all 3 files - SHARED_PROMPT_PATTERNS: {id}.analysis.md → {dok_id}-analysis.md - news-realtime-monitor: {dok_id}.analysis.md → {dok_id}-analysis.md - news-evening-analysis: {id}.analysis.md → {dok_id}-analysis.md 3. Expanded evening-analysis quality gate from 3 checks to 7 checks matching SHARED_PROMPT_PATTERNS (added: style directives, SWOT evidence tables, structured tables, coverage vs JSON count) Agent-Logs-Url: https://github.com/Hack23/riksdagsmonitor/sessions/b81b556e-59b9-49c8-b0d2-1744b795d910 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
diff --git a/.github/workflows/SHARED_PROMPT_PATTERNS.md b/.github/workflows/SHARED_PROMPT_PATTERNS.md
@@ -630,7 +630,7 @@ Read these methodology documents to guide your analysis:
    - STRIDE threat analysis (where applicable)
    - Stakeholder impact matrix (6 lenses)
    - Forward indicators (specific watch items)
-3. **Write analysis:** Save as `{id}.analysis.md` alongside the data file
+3. **Write analysis:** Save as `{dok_id}-analysis.md` alongside the data file
 4. **Include Mermaid diagrams** — at least 1 per file, color-coded:
    ```
    style X fill:#dc3545,color:#fff   /* Red — critical */
diff --git a/.github/workflows/news-evening-analysis.md b/.github/workflows/news-evening-analysis.md
@@ -699,7 +699,7 @@ For each pending file from the catalog (ordered by significance — propositions
 7. **Stakeholders** — 6-lens impact matrix (government, opposition, citizen, economic, international, media)
 8. **Forward indicators** — Specific watch items with concrete timelines and triggers
 9. **Mermaid diagrams** — At least 1 diagram with REAL data from the file (not placeholder text)
-10. **Write** `{id}.analysis.md` alongside the data file
+10. **Write** `{dok_id}-analysis.md` alongside the data file
 
 **Quality standard:** Each analysis file must match [SWOT.md](../../SWOT.md) / [THREAT_MODEL.md](../../THREAT_MODEL.md) quality — Hack23 header badges, color-coded Mermaid diagrams, evidence tables with confidence labels, and actionable intelligence.
 
@@ -771,6 +771,7 @@ fi
 ANALYSIS_DIR="analysis/daily/$ARTICLE_DATE"
 QUALITY_PASS=true
 FAIL_COUNT=0
+WARN_COUNT=0
 
 echo "=== 🔍 Analysis Quality Gate Check ==="
 
@@ -782,28 +783,62 @@ echo "📊 Daily: $(echo "$DAILY_MD_FILES" | grep -c '.' 2>/dev/null || true) |
 # Check 1: Daily synthesis Mermaid diagrams
 for f in $DAILY_MD_FILES; do
   [ ! -f "$f" ] && continue
-  if [ "$(grep -c '```mermaid' "$f" 2>/dev/null || true)" -eq 0 ]; then
+  MERMAID_COUNT=$(grep -c '```mermaid' "$f" 2>/dev/null || true)
+  if [ "${MERMAID_COUNT:-0}" -eq 0 ]; then
     echo "❌ FAIL: $(basename "$f") has NO Mermaid diagrams"
     QUALITY_PASS=false; FAIL_COUNT=$((FAIL_COUNT + 1))
   fi
 done
 
-# Check 2: No [REQUIRED] placeholders
+# Check 2: Color-coded style directives in Mermaid diagrams
+for f in $DAILY_MD_FILES; do
+  [ ! -f "$f" ] && continue
+  if grep -q '```mermaid' "$f" 2>/dev/null; then
+    STYLE_COUNT=$(grep -c 'style.*fill:#' "$f" 2>/dev/null || true)
+    if [ "${STYLE_COUNT:-0}" -eq 0 ]; then
+      echo "❌ FAIL: $(basename "$f") Mermaid has NO color-coded style directives"
+      QUALITY_PASS=false; FAIL_COUNT=$((FAIL_COUNT + 1))
+    fi
+  fi
+done
+
+# Check 3: No [REQUIRED] placeholders
 for f in $ALL_MD_FILES; do
   [ ! -f "$f" ] && continue
-  if [ "$(grep -c '\[REQUIRED\]' "$f" 2>/dev/null || true)" -gt 0 ]; then
-    echo "❌ FAIL: $(basename "$f") has unfilled [REQUIRED] placeholders"
+  REQ_COUNT=$(grep -c '\[REQUIRED\]' "$f" 2>/dev/null || true)
+  if [ "${REQ_COUNT:-0}" -gt 0 ]; then
+    echo "❌ FAIL: $(basename "$f") has $REQ_COUNT unfilled [REQUIRED] placeholders"
     QUALITY_PASS=false; FAIL_COUNT=$((FAIL_COUNT + 1))
   fi
 done
 
-# Check 3: Per-file analyses must NOT be stubs
+# Check 4: SWOT evidence tables with dok_id
+SWOT_FILE="$ANALYSIS_DIR/swot-analysis.md"
+if [ -f "$SWOT_FILE" ]; then
+  TABLE_COUNT=$(grep -c '|.*dok_id\||.*Evidence' "$SWOT_FILE" 2>/dev/null || true)
+  if [ "${TABLE_COUNT:-0}" -eq 0 ]; then
+    echo "❌ FAIL: swot-analysis.md has NO evidence tables with dok_id"
+    QUALITY_PASS=false; FAIL_COUNT=$((FAIL_COUNT + 1))
+  fi
+fi
+
+# Check 5: Structured tables in daily synthesis (not just plain prose)
+for f in $DAILY_MD_FILES; do
+  [ ! -f "$f" ] && continue
+  TABLE_COUNT=$(grep -c '^|' "$f" 2>/dev/null || true)
+  if [ "${TABLE_COUNT:-0}" -lt 3 ]; then
+    echo "⚠️ WARNING: $(basename "$f") has only $TABLE_COUNT table rows — templates require structured tables"
+    WARN_COUNT=$((WARN_COUNT + 1))
+  fi
+done
+
+# Check 6: Per-file analyses must NOT be stubs
 STUB_COUNT=0
 for f in $PERFILE_MD_FILES; do
   [ ! -f "$f" ] && continue
   STUB_SCORE=0
   [ "$(grep -cE '_No (strengths|weaknesses|opportunities|threats) identified_' "$f" 2>/dev/null || true)" -ge 2 ] && STUB_SCORE=$((STUB_SCORE + 2))
-  [ "$(grep -c 'this document requires assessment of\|this document warrants scrutiny for\|this document may affect business' "$f" 2>/dev/null || true)" -ge 2 ] && STUB_SCORE=$((STUB_SCORE + 2))
+  [ "$(grep -c 'this document requires assessment of\|this document warrants scrutiny for\|this document may affect business\|this document has low newsworthiness\|this document must be assessed for' "$f" 2>/dev/null || true)" -ge 2 ] && STUB_SCORE=$((STUB_SCORE + 2))
   [ "$(grep -c '```mermaid' "$f" 2>/dev/null || true)" -eq 0 ] && STUB_SCORE=$((STUB_SCORE + 1))
   [ "$(grep -c '^|' "$f" 2>/dev/null || true)" -lt 2 ] && STUB_SCORE=$((STUB_SCORE + 1))
   if [ "${STUB_SCORE:-0}" -ge 3 ]; then
@@ -812,12 +847,29 @@ for f in $PERFILE_MD_FILES; do
   fi
 done
 
+# Check 7: Coverage — every JSON must have an analysis
+if [ -d "$ANALYSIS_DIR/documents" ]; then
+  JSON_COUNT=$(find "$ANALYSIS_DIR/documents" -name "*.json" -type f 2>/dev/null | wc -l)
+  ANALYSIS_MD_COUNT=$(find "$ANALYSIS_DIR/documents" -name "*-analysis.md" -type f 2>/dev/null | wc -l)
+  if [ "${JSON_COUNT:-0}" -gt 0 ] && [ "${ANALYSIS_MD_COUNT:-0}" -lt "${JSON_COUNT:-0}" ]; then
+    echo "❌ FAIL: Only $ANALYSIS_MD_COUNT analysis files for $JSON_COUNT data files"
+    QUALITY_PASS=false; FAIL_COUNT=$((FAIL_COUNT + 1))
+  elif [ "${JSON_COUNT:-0}" -gt 0 ]; then
+    echo "✅ PASS: $ANALYSIS_MD_COUNT analysis files for $JSON_COUNT data files"
+  fi
+fi
+
 echo ""
+echo "=== Quality Gate Summary ==="
+echo "Failures: $FAIL_COUNT | Warnings: $WARN_COUNT"
 if [ "$QUALITY_PASS" = "true" ]; then
   echo "✅ Quality gate PASSED — proceed to article generation"
 else
   echo "❌ Quality gate FAILED ($FAIL_COUNT failures) — fix analysis files before proceeding"
   [ "${STUB_COUNT:-0}" -gt 0 ] && echo "🚨 $STUB_COUNT per-file analyses are stubs — read analysis/templates/per-file-political-intelligence.md and rewrite"
+  echo "📌 For per-file analyses: read analysis/templates/per-file-political-intelligence.md"
+  echo "📌 For daily synthesis: read the corresponding template in analysis/templates/"
+  echo "📌 Reference good examples: SWOT.md, THREAT_MODEL.md"
 fi
 ```
 
@@ -827,7 +879,7 @@ fi
 
 **Before deciding whether to generate articles or call noop, you MUST:**
 
-1. **Review the analysis artifacts** in `analysis/daily/YYYY-MM-DD/` and per-file `.analysis.md` files — read `synthesis-summary.md` and significance scores to understand what was found
+1. **Review the analysis artifacts** in `analysis/daily/YYYY-MM-DD/` and per-file `-analysis.md` files — read `synthesis-summary.md` and significance scores to understand what was found
 2. **Summarize the analysis findings** — note how many documents were downloaded, their significance scores, key themes, and risk levels
 3. **ALWAYS commit analysis artifacts** regardless of whether articles will be generated:
 
@@ -846,7 +898,7 @@ else
 fi
 ```
 
-> **🚨 CRITICAL RULE: Never call `safeoutputs___noop` if analysis artifacts exist.** If the analysis produced ANY output files (per-file `.analysis.md` or daily synthesis), you MUST commit them via `safeoutputs___create_pull_request` — even if no articles are generated. Use an analysis-only PR with title: `📊 Analysis Only - Evening Analysis - {date}` and label `analysis-only`. Only use `safeoutputs___noop` if NO analysis output was generated.
+> **🚨 CRITICAL RULE: Never call `safeoutputs___noop` if analysis artifacts exist.** If the analysis produced ANY output files (per-file `-analysis.md` or daily synthesis), you MUST commit them via `safeoutputs___create_pull_request` — even if no articles are generated. Use an analysis-only PR with title: `📊 Analysis Only - Evening Analysis - {date}` and label `analysis-only`. Only use `safeoutputs___noop` if NO analysis output was generated.
 
 ## Step 2: Gather Parliamentary Data
 
diff --git a/.github/workflows/news-realtime-monitor.md b/.github/workflows/news-realtime-monitor.md
@@ -295,7 +295,7 @@ fi
    - `analysis/templates/stakeholder-impact.md`
    - `analysis/templates/significance-scoring.md`
 
-3. **For EVERY downloaded document/data file**: apply ALL 6 analytical lenses and create `{dok_id}.analysis.md` following the per-file template. Cite specific data (dok_id, vote counts, party names). Include ≥1 color-coded Mermaid diagram with `style` directives.
+3. **For EVERY downloaded document/data file**: apply ALL 6 analytical lenses and create `{dok_id}-analysis.md` following the per-file template. Cite specific data (dok_id, vote counts, party names). Include ≥1 color-coded Mermaid diagram with `style` directives.
 
 4. **Create/rewrite ALL 7 daily synthesis files** in `analysis/daily/$ARTICLE_DATE/` — each MUST follow its template EXACTLY (metadata header, Mermaid diagrams with color-coded style directives, structured evidence tables, confidence labels, no `[REQUIRED]` placeholders).
 
@@ -392,16 +392,18 @@ echo "📊 Total pending per-file analysis files (all dates): $PENDING"
 After data is downloaded, you MUST complete ALL of these steps before proceeding to event detection:
 
 **Step A — Read templates and methodologies** (FIRST, before writing anything):
-1. `view analysis/templates/per-file-political-intelligence.md` — read FULLY, note the required structure
-2. `view analysis/methodologies/ai-driven-analysis-guide.md` — read the "BAD vs GOOD" examples
-3. `view analysis/methodologies/political-swot-framework.md` — understand evidence tables
+1. Follow the organization-wide **SHARED_PROMPT_PATTERNS Step 2 + Step 3** exactly: read **all 6 methodology guides** and **all 8 analysis templates** defined there (in `analysis/methodologies/` and `analysis/templates/`) **before writing any analysis**. Do NOT subset or skip any required document.
+2. After completing SHARED_PROMPT_PATTERNS Steps 2–3, (re)read these **news-monitor-specific assets**:
+   - `view analysis/templates/per-file-political-intelligence.md` — read FULLY, note the required structure
+   - `view analysis/methodologies/ai-driven-analysis-guide.md` — read the "BAD vs GOOD" examples
+   - `view analysis/methodologies/political-swot-framework.md` — understand evidence tables
 
 **Step B — Create real per-file analyses** (for EVERY document):
 1. List all downloaded documents: `find analysis/daily/${ARTICLE_DATE:-$(date -u +%Y-%m-%d)}/documents/ -name "*.json" -type f`
 2. For EACH JSON file:
    a. Read it with `view` — extract dok_id, titel, datum, parti, organ
    b. Apply ALL 6 analytical lenses (classification, SWOT, risk, STRIDE, stakeholders, forward indicators)
-   c. Write or rewrite `{dok_id}-analysis.md` following the per-file template EXACTLY
+   c. Write or rewrite the per-file analysis markdown so that its filename matches the `*-analysis.md` convention (for example `{dok_id}-analysis.md`) and follows the per-file template EXACTLY
    d. Include ≥1 color-coded Mermaid diagram with `style` directives and REAL data
    e. Include structured evidence tables with dok_id, confidence, impact columns
    f. SWOT quadrants must have REAL entries — NOT "_No strengths identified_"
@@ -529,7 +531,7 @@ fi
 **Before deciding whether to generate articles or call noop, you MUST:**
 
 1. **Verify data was downloaded** — `find analysis/data/ -name "*.json" -type f | wc -l` must be > 0
-2. **Verify analysis was created** — every downloaded document has a `.analysis.md` file
+2. **Verify analysis was created** — every downloaded document has a `-analysis.md` file
 3. **Verify daily synthesis files follow templates** — no `[REQUIRED]` placeholders, Mermaid diagrams with real data
 4. **ALWAYS commit data AND analysis together**:
 
