Skip to content

Prevent repeated workflow runs from overwriting analysis and articles via slug-level suffix deduplication#915

Merged
pethers merged 18 commits intomainfrom
copilot/analyze-daily-content-news-2026-04-02
Apr 3, 2026
Merged

Prevent repeated workflow runs from overwriting analysis and articles via slug-level suffix deduplication#915
pethers merged 18 commits intomainfrom
copilot/analyze-daily-content-news-2026-04-02

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 2, 2026

Repeated same-day workflow runs (e.g. breaking news every 6h) silently overwrote prior analysis directories and news articles, destroying committed intelligence. Additionally, the previous PR's improvements to AI analysis guides, workflow prompts, and article template analysis links are included.

Slug-level suffix deduplication

Two new utilities in src/utils/file-utils.ts:

  • resolveUniqueAnalysisDir(baseDir) — uses a claimAnalysisDir() helper with atomic fs.mkdirSync({ recursive: false }) and EEXIST retry to race-safely claim suffixed directories. The base directory uses manifest.json presence to determine occupancy (supporting incremental/skipCompleted runs), while suffixed candidates (-2, -3, …) use atomic mkdirSync claiming. UUID-based fallback (randomUUID().slice(0,8)) for overflow.
  • resolveUniqueFilePath(filepath) — appends -2, -3, … before the extension when the file exists, with UUID-based fallback.

Deduplication operates at the slug level, not the file level. The dedup suffix is computed by comparing the resolved analysis directory basename with the original base slug, then passed separately as dedupSuffix (e.g. "-2") to each strategy. This ensures multiple article types in one run retain distinct slugs (e.g. breaking-2, week-ahead-2) without multi-type slug corruption. The suffix is validated against a dedup pattern (/^-[\da-f]+$/i) to support both numeric and UUID formats. The suffixed slug is baked into the article before HTML generation, so filenames, canonical URLs, og:url, and language-switcher links all stay consistent:

analysis/2026-04-02/breaking/           ← first run
analysis/2026-04-02/breaking-2/         ← second run (preserved, not overwritten)

news/2026-04-02-breaking-en.html        ← first run
news/2026-04-02-breaking-2-en.html      ← second run (consistent slug)

The existing ARTICLE_FILENAME_PATTERN regex matches deduplicated filenames without modification, since the -2 suffix falls within the slug capture group (.+).

  • analysis-stage.ts: validates hasSubstantiveData(fetchedData) before calling resolveUniqueAnalysisDir to prevent orphan claimed directories on early abort; resolves unique dir via atomic claimAnalysisDir before ensureDirectoryExists
  • generate-stage.ts: deriveTypeSlug(strategyType, dedupSuffix) helper appends the dedup suffix to each strategy type independently, keeping per-strategy slugs distinct; passes the suffixed slug as both slug and analysisDir into generateArticleHTML() so canonical/og:url/language-switcher links match the written filename
  • output-stage.ts: writeArticleFile returns the actual written filename (string | null) instead of boolean, with no file-level dedup (slug-level dedup upstream ensures uniqueness); JSDoc aligned to reflect actual behavior
  • news-enhanced.ts: computes dedupSuffix by comparing resolved dir basename with original slug, passes both dedupSuffix and analysisDir per-strategy through generateArticleForStrategy()

Analysis directory provenance in article links

  • ArticleOptions.analysisDir — new optional field that overrides the slug when constructing Analysis & Transparency links, so articles generated from a suffixed analysis run (e.g. breaking-2) link to the correct analysis directory instead of the original
  • renderAnalysisTransparencySection accepts and uses the resolved analysis directory name through the pipeline
  • End-to-end propagationnews-enhanced.ts captures AnalysisContext from runAnalysisWithGuard(), extracts the resolved directory basename via path.basename(analysisCtx.outputDir), and passes it as analysisDir through generateArticleForStrategy()generateSingleLanguageArticle()generateArticleHTML()renderAnalysisTransparencySection(), ensuring articles always link to the actual analysis output directory
  • manifest.json link — Analysis & Transparency section now includes a direct link to manifest.json for run provenance per AI Analysis Guide Rule 10

Localized per-file analysis link labels

All 18 individual analysis file links (classification, threat assessment, risk scoring, deep analysis sections) now use fully localized labels across all 14 supported languages via getLocalizedString() and dedicated LanguageMap constants in language-ui.ts, fixing the English-only regression for non-English articles and screen readers.

Unit test coverage for dedup behavior

Added 11 focused Vitest cases covering:

  • deriveTypeSlug with numeric suffixes, UUID suffixes, empty suffix, and per-strategy distinctness
  • generateArticleForStrategy with dedup suffix propagation to filenames/slugs
  • resolveUniqueAnalysisDir base dir behavior, suffixed candidate claiming, manifest.json occupancy check, and no-manifest reuse

AI analysis quality improvements (from prior session)

  • AI Analysis Guide v3.0 — 4 new rules (9–12): AI-driven headlines, comprehensive analysis-to-article linkage, script analysis replaced with AI prompt placeholders, zero script-generated prose. Dedup filename example updated to match slug-level scheme (breaking-2-en.html, not breaking-en-2.html).
  • Article template — Analysis & Transparency section links to all 18 individual analysis .md files plus manifest.json instead of 5 directory-level links
  • Analysis stage — 5 existing/ methods (deep-analysis, stakeholder, coalition, voting-patterns, cross-session) now emit structured AI prompt templates instead of bare data inventory tables
  • Workflow prompts — All 4 news workflows gain mandatory 📰 AI-DRIVEN HEADLINE AND DESCRIPTION GENERATION sections with rejected/required title examples

Copilot AI and others added 4 commits April 2, 2026 22:36
…e analysis links

- Add Rules 9-12 to ai-driven-analysis-guide.md for AI-driven headlines, analysis linkage, prompt placeholders, and narrative separation
- Update article template to link to all 18 individual analysis files instead of just directory-level links
- Transform 5 existing analysis methods to produce AI prompt placeholders instead of bare data tables
- Add AI-driven headline/description generation sections to all 4 news workflows (breaking, committee-reports, motions, propositions)
- Add analysis file reference verification checklists to all workflows

Agent-Logs-Url: https://github.com/Hack23/euparliamentmonitor/sessions/63626a43-654f-44ed-8da2-135dc82cb84e

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
…ticles on repeated runs

- Add resolveUniqueAnalysisDir() to file-utils.ts: appends -2, -3, etc. when manifest.json exists
- Add resolveUniqueFilePath() to file-utils.ts: appends -2, -3, etc. to article filenames
- Update analysis-stage.ts to use resolveUniqueAnalysisDir before creating output dir
- Update output-stage.ts writeArticleFile to use resolveUniqueFilePath instead of overwriting
- Update ai-driven-analysis-guide.md Rule 1 to document suffix deduplication behavior
- Update tests to match new non-overwrite behavior

Agent-Logs-Url: https://github.com/Hack23/euparliamentmonitor/sessions/ef87e601-17be-49e0-81d8-5c34cdc34cd5

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
@pethers pethers marked this pull request as ready for review April 2, 2026 23:21
@github-actions github-actions bot added documentation Documentation updates infrastructure CI/CD and build infrastructure github_actions GitHub Actions updates testing Testing improvements news News generation and articles test-coverage-impact Impacts test coverage needs-tests Needs test coverage workflows GitHub workflows scripts Scripts and automation labels Apr 2, 2026
@pethers pethers requested a review from Copilot April 2, 2026 23:21
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Prevents repeated same-day workflow runs from overwriting prior analysis outputs and generated news articles by introducing suffix-based deduplication, and expands AI analysis guidance/prompts plus article transparency linking to individual analysis artifacts.

Changes:

  • Added utilities to resolve unique analysis directories and unique article file paths via numeric suffixing (-2, -3, …).
  • Updated pipeline stages to use the new deduplication utilities and adjusted unit tests accordingly.
  • Expanded AI analysis templates/prompts and updated article templates + workflows to require AI-driven headlines/descriptions and link to individual analysis files.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
src/utils/file-utils.ts Adds resolveUniqueAnalysisDir() and resolveUniqueFilePath() utilities.
src/generators/pipeline/analysis-stage.ts Uses resolveUniqueAnalysisDir() to avoid overwriting prior analysis runs.
src/generators/pipeline/output-stage.ts Uses resolveUniqueFilePath() to avoid overwriting existing articles.
src/templates/article-template.ts Expands Analysis & Transparency section to link to individual analysis .md files.
scripts/utils/file-utils.js Compiled JS equivalent of new file util functions.
scripts/generators/pipeline/analysis-stage.js Compiled JS equivalent of analysis-stage deduplication changes.
scripts/generators/pipeline/output-stage.js Compiled JS equivalent of output-stage deduplication changes.
scripts/templates/article-template.js Compiled JS equivalent of expanded transparency links.
test/unit/pipeline-stages.test.js Updates expectations: no overwrite; writes suffixed article filenames.
test/unit/analysis-stage.test.js Updates expectations: second run writes to suffixed analysis directory / adjusted assertions.
analysis/methodologies/ai-driven-analysis-guide.md Updates guide to v3.0 including dedup + headline/linkage rules.
.github/workflows/news-breaking.md Adds mandatory AI-driven headline/description + analysis link verification guidance.
.github/workflows/news-committee-reports.md Adds mandatory AI-driven headline/description + analysis link verification guidance.
.github/workflows/news-motions.md Adds mandatory AI-driven headline/description + analysis link verification guidance.
.github/workflows/news-propositions.md Adds mandatory AI-driven headline/description + analysis link verification guidance.
Comments suppressed due to low confidence (1)

scripts/generators/pipeline/output-stage.js:71

  • writeArticleFile() may write to a suffixed filename via resolveUniqueFilePath(), but it still returns only a boolean. writeSingleArticle() then pushes the original filename into stats.articles, so the generation metadata can reference files that were never written (when deduplication occurs). Consider returning the actual written filename/path from writeArticleFile() and using it when updating stats.
    // When not in skip mode: resolve a unique path to avoid overwriting
    // existing articles from prior workflow runs on the same date.
    const uniquePath = resolveUniqueFilePath(filepath);
    atomicWrite(uniquePath, html);
    const writtenName = path.basename(uniquePath);
    if (writtenName !== filename) {
        console.log(`  ✅ Wrote: ${writtenName} (unique — original ${filename} already existed)`);
    }
    else {
        console.log(`  ✅ Wrote: ${filename}`);
    }
    return true;
}
/**
 * Write a language-specific article file and update the generation stats.
 *
 * @param html - Full HTML content to write
 * @param slug - Article slug (e.g. `"2025-01-15-week-ahead"`)
 * @param lang - Language code suffix (e.g. `"en"`)
 * @param options - Output flags and directory path
 * @param stats - Mutable stats object to increment counters on
 * @returns `true` when the file was actually written
 */
export function writeSingleArticle(html, slug, lang, options, stats) {
    const filename = `${slug}-${lang}.html`;
    const written = writeArticleFile(html, filename, options);
    if (written) {
        stats.generated += 1;
        stats.articles.push(filename);
    }

Comment on lines +62 to +71
// When not in skip mode: resolve a unique path to avoid overwriting
// existing articles from prior workflow runs on the same date.
const uniquePath = resolveUniqueFilePath(filepath);
atomicWrite(uniquePath, html);
const writtenName = path.basename(uniquePath);
if (writtenName !== filename) {
console.log(` ✅ Wrote: ${writtenName} (unique — original ${filename} already existed)`);
} else {
console.log(` ✅ Wrote: ${filename}`);
}
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

writeArticleFile now writes to a potentially suffixed filename via resolveUniqueFilePath(), but it still only returns a boolean. Callers like writeSingleArticle() (same module) record filename into stats.articles, so a deduplicated write will cause metadata/stats to reference a file that was never created. Consider returning the actual written filename/path (or exposing it via an out param) and updating stats/logging to use that value.

Copilot uses AI. Check for mistakes.
Comment on lines +2719 to +2725
// resolveUniqueAnalysisDir appends a numeric suffix (-2, -3, …) when
// a prior completed run (with manifest.json) already occupies the path,
// preventing repeated workflow runs from overwriting previous analysis.
const preferredDir = articleTypeSlug
? path.resolve(outputDir, date, articleTypeSlug)
: path.resolve(outputDir, date);
const dateOutputDir = resolveUniqueAnalysisDir(preferredDir);
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolveUniqueAnalysisDir(preferredDir) can produce a suffixed directory like analysis/{date}/{slug}-2, but the article transparency links are built from {date} + {slug} only. On repeated same-day runs, newly generated articles will link to the original analysis directory instead of the run-specific suffixed one, breaking provenance. Consider passing the resolved analysis directory name (from AnalysisContext.outputDir or the manifest) into generateArticleHTML() / renderAnalysisTransparencySection() so links match the actual output path.

Copilot uses AI. Check for mistakes.
Comment on lines 525 to 569
const repoBase = 'https://github.com/Hack23/euparliamentmonitor/blob/main';
const treeDirBase = 'https://github.com/Hack23/euparliamentmonitor/tree/main';
const analysisDir = `${treeDirBase}/analysis/${safeDate}/${safeSlug}`;
const analysisFileBase = `${repoBase}/analysis/${safeDate}/${safeSlug}`;
const methodologyDir = `${repoBase}/analysis/methodologies`;

return `
<section class="analysis-transparency" aria-label="${heading}">
<h2 id="analysis-transparency-heading">${heading}</h2>
<p>${disclosure}</p>
<nav class="analysis-links" aria-labelledby="analysis-transparency-heading">
<h3><span aria-hidden="true">📊</span> ${analysisSummaryLabel}</h3>
<ul>
<li><a href="${analysisDir}" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">📊</span> ${analysisSummaryLabel}</a></li>
<li><a href="${analysisDir}/classification" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">🏷️</span> ${classificationLabel}</a></li>
<li><a href="${analysisDir}/threat-assessment" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">🛡️</span> ${threatLabel}</a></li>
<li><a href="${analysisDir}/risk-scoring" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">⚖️</span> ${riskLabel}</a></li>
<li><a href="${analysisDir}/existing" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">🔍</span> ${deepLabel}</a></li>
<li><a href="${analysisDir}" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">📁</span> ${analysisSummaryLabel}</a></li>
</ul>
<h3><span aria-hidden="true">🏷️</span> ${classificationLabel}</h3>
<ul>
<li><a href="${analysisFileBase}/classification/significance-assessment.md" target="_blank" rel="noopener noreferrer">Significance Assessment</a></li>
<li><a href="${analysisFileBase}/classification/actor-mapping.md" target="_blank" rel="noopener noreferrer">Actor Mapping</a></li>
<li><a href="${analysisFileBase}/classification/forces-analysis.md" target="_blank" rel="noopener noreferrer">Forces Analysis</a></li>
<li><a href="${analysisFileBase}/classification/impact-matrix.md" target="_blank" rel="noopener noreferrer">Impact Matrix</a></li>
</ul>
<h3><span aria-hidden="true">🛡️</span> ${threatLabel}</h3>
<ul>
<li><a href="${analysisFileBase}/threat-assessment/political-threat-landscape.md" target="_blank" rel="noopener noreferrer">Political Threat Landscape</a></li>
<li><a href="${analysisFileBase}/threat-assessment/actor-threat-profiles.md" target="_blank" rel="noopener noreferrer">Actor Threat Profiles</a></li>
<li><a href="${analysisFileBase}/threat-assessment/consequence-trees.md" target="_blank" rel="noopener noreferrer">Consequence Trees</a></li>
<li><a href="${analysisFileBase}/threat-assessment/legislative-disruption.md" target="_blank" rel="noopener noreferrer">Legislative Disruption</a></li>
</ul>
<h3><span aria-hidden="true">⚖️</span> ${riskLabel}</h3>
<ul>
<li><a href="${analysisFileBase}/risk-scoring/risk-matrix.md" target="_blank" rel="noopener noreferrer">Risk Matrix</a></li>
<li><a href="${analysisFileBase}/risk-scoring/quantitative-swot.md" target="_blank" rel="noopener noreferrer">Quantitative SWOT</a></li>
<li><a href="${analysisFileBase}/risk-scoring/political-capital-risk.md" target="_blank" rel="noopener noreferrer">Political Capital Risk</a></li>
<li><a href="${analysisFileBase}/risk-scoring/legislative-velocity-risk.md" target="_blank" rel="noopener noreferrer">Legislative Velocity Risk</a></li>
<li><a href="${analysisFileBase}/risk-scoring/agent-risk-workflow.md" target="_blank" rel="noopener noreferrer">Agent Risk Workflow</a></li>
</ul>
<h3><span aria-hidden="true">🔍</span> ${deepLabel}</h3>
<ul>
<li><a href="${analysisFileBase}/existing/deep-analysis.md" target="_blank" rel="noopener noreferrer">Deep Analysis</a></li>
<li><a href="${analysisFileBase}/existing/stakeholder-analysis.md" target="_blank" rel="noopener noreferrer">Stakeholder Analysis</a></li>
<li><a href="${analysisFileBase}/existing/coalition-analysis.md" target="_blank" rel="noopener noreferrer">Coalition Analysis</a></li>
<li><a href="${analysisFileBase}/existing/voting-patterns.md" target="_blank" rel="noopener noreferrer">Voting Patterns</a></li>
<li><a href="${analysisFileBase}/existing/cross-session-intelligence.md" target="_blank" rel="noopener noreferrer">Cross-Session Intelligence</a></li>
</ul>
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This section hard-codes analysis links to analysis/{date}/{slug}. With suffix deduplication in the analysis stage (e.g. analysis/{date}/{slug}-2), repeated workflow runs will produce articles whose Analysis & Transparency links point to the wrong (earlier) analysis artifacts. The template likely needs an additional parameter (e.g. analysisSlug / analysisDirName) so it can link to the resolved analysis output directory for the specific run.

Copilot uses AI. Check for mistakes.
Comment on lines +540 to +568
<h3><span aria-hidden="true">🏷️</span> ${classificationLabel}</h3>
<ul>
<li><a href="${analysisFileBase}/classification/significance-assessment.md" target="_blank" rel="noopener noreferrer">Significance Assessment</a></li>
<li><a href="${analysisFileBase}/classification/actor-mapping.md" target="_blank" rel="noopener noreferrer">Actor Mapping</a></li>
<li><a href="${analysisFileBase}/classification/forces-analysis.md" target="_blank" rel="noopener noreferrer">Forces Analysis</a></li>
<li><a href="${analysisFileBase}/classification/impact-matrix.md" target="_blank" rel="noopener noreferrer">Impact Matrix</a></li>
</ul>
<h3><span aria-hidden="true">🛡️</span> ${threatLabel}</h3>
<ul>
<li><a href="${analysisFileBase}/threat-assessment/political-threat-landscape.md" target="_blank" rel="noopener noreferrer">Political Threat Landscape</a></li>
<li><a href="${analysisFileBase}/threat-assessment/actor-threat-profiles.md" target="_blank" rel="noopener noreferrer">Actor Threat Profiles</a></li>
<li><a href="${analysisFileBase}/threat-assessment/consequence-trees.md" target="_blank" rel="noopener noreferrer">Consequence Trees</a></li>
<li><a href="${analysisFileBase}/threat-assessment/legislative-disruption.md" target="_blank" rel="noopener noreferrer">Legislative Disruption</a></li>
</ul>
<h3><span aria-hidden="true">⚖️</span> ${riskLabel}</h3>
<ul>
<li><a href="${analysisFileBase}/risk-scoring/risk-matrix.md" target="_blank" rel="noopener noreferrer">Risk Matrix</a></li>
<li><a href="${analysisFileBase}/risk-scoring/quantitative-swot.md" target="_blank" rel="noopener noreferrer">Quantitative SWOT</a></li>
<li><a href="${analysisFileBase}/risk-scoring/political-capital-risk.md" target="_blank" rel="noopener noreferrer">Political Capital Risk</a></li>
<li><a href="${analysisFileBase}/risk-scoring/legislative-velocity-risk.md" target="_blank" rel="noopener noreferrer">Legislative Velocity Risk</a></li>
<li><a href="${analysisFileBase}/risk-scoring/agent-risk-workflow.md" target="_blank" rel="noopener noreferrer">Agent Risk Workflow</a></li>
</ul>
<h3><span aria-hidden="true">🔍</span> ${deepLabel}</h3>
<ul>
<li><a href="${analysisFileBase}/existing/deep-analysis.md" target="_blank" rel="noopener noreferrer">Deep Analysis</a></li>
<li><a href="${analysisFileBase}/existing/stakeholder-analysis.md" target="_blank" rel="noopener noreferrer">Stakeholder Analysis</a></li>
<li><a href="${analysisFileBase}/existing/coalition-analysis.md" target="_blank" rel="noopener noreferrer">Coalition Analysis</a></li>
<li><a href="${analysisFileBase}/existing/voting-patterns.md" target="_blank" rel="noopener noreferrer">Voting Patterns</a></li>
<li><a href="${analysisFileBase}/existing/cross-session-intelligence.md" target="_blank" rel="noopener noreferrer">Cross-Session Intelligence</a></li>
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new per-file analysis links use English-only link text (e.g. “Significance Assessment”, “Actor Mapping”) even when generating non-English articles (<article lang="...">). Since the surrounding headings/labels are localized, this is a regression for multilingual UX and screen readers. Consider localizing these link titles (or at least reusing existing localized labels where they exist) so the section language matches the document language.

Copilot uses AI. Check for mistakes.
Comment on lines 2045 to +2053
// When articleTypeSlug is provided, scope output to a per-article-type
// subdirectory so concurrent workflows on the same date never collide.
const dateOutputDir = articleTypeSlug
// resolveUniqueAnalysisDir appends a numeric suffix (-2, -3, …) when
// a prior completed run (with manifest.json) already occupies the path,
// preventing repeated workflow runs from overwriting previous analysis.
const preferredDir = articleTypeSlug
? path.resolve(outputDir, date, articleTypeSlug)
: path.resolve(outputDir, date);
const dateOutputDir = resolveUniqueAnalysisDir(preferredDir);
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolveUniqueAnalysisDir(preferredDir) can return a suffixed path (e.g. analysis/{date}/{slug}-2), but article generation and linking logic elsewhere appears to still key off {date} + {slug}. That means repeated same-day runs can generate new analysis in a suffixed directory while produced articles still point to / consume the original directory, breaking run provenance. Consider propagating the resolved output directory (or its basename) through the pipeline so consumers link to the correct run folder.

Copilot uses AI. Check for mistakes.
Comment on lines 413 to 456
const repoBase = 'https://github.com/Hack23/euparliamentmonitor/blob/main';
const treeDirBase = 'https://github.com/Hack23/euparliamentmonitor/tree/main';
const analysisDir = `${treeDirBase}/analysis/${safeDate}/${safeSlug}`;
const analysisFileBase = `${repoBase}/analysis/${safeDate}/${safeSlug}`;
const methodologyDir = `${repoBase}/analysis/methodologies`;
return `
<section class="analysis-transparency" aria-label="${heading}">
<h2 id="analysis-transparency-heading">${heading}</h2>
<p>${disclosure}</p>
<nav class="analysis-links" aria-labelledby="analysis-transparency-heading">
<h3><span aria-hidden="true">📊</span> ${analysisSummaryLabel}</h3>
<ul>
<li><a href="${analysisDir}" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">📊</span> ${analysisSummaryLabel}</a></li>
<li><a href="${analysisDir}/classification" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">🏷️</span> ${classificationLabel}</a></li>
<li><a href="${analysisDir}/threat-assessment" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">🛡️</span> ${threatLabel}</a></li>
<li><a href="${analysisDir}/risk-scoring" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">⚖️</span> ${riskLabel}</a></li>
<li><a href="${analysisDir}/existing" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">🔍</span> ${deepLabel}</a></li>
<li><a href="${analysisDir}" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">📁</span> ${analysisSummaryLabel}</a></li>
</ul>
<h3><span aria-hidden="true">🏷️</span> ${classificationLabel}</h3>
<ul>
<li><a href="${analysisFileBase}/classification/significance-assessment.md" target="_blank" rel="noopener noreferrer">Significance Assessment</a></li>
<li><a href="${analysisFileBase}/classification/actor-mapping.md" target="_blank" rel="noopener noreferrer">Actor Mapping</a></li>
<li><a href="${analysisFileBase}/classification/forces-analysis.md" target="_blank" rel="noopener noreferrer">Forces Analysis</a></li>
<li><a href="${analysisFileBase}/classification/impact-matrix.md" target="_blank" rel="noopener noreferrer">Impact Matrix</a></li>
</ul>
<h3><span aria-hidden="true">🛡️</span> ${threatLabel}</h3>
<ul>
<li><a href="${analysisFileBase}/threat-assessment/political-threat-landscape.md" target="_blank" rel="noopener noreferrer">Political Threat Landscape</a></li>
<li><a href="${analysisFileBase}/threat-assessment/actor-threat-profiles.md" target="_blank" rel="noopener noreferrer">Actor Threat Profiles</a></li>
<li><a href="${analysisFileBase}/threat-assessment/consequence-trees.md" target="_blank" rel="noopener noreferrer">Consequence Trees</a></li>
<li><a href="${analysisFileBase}/threat-assessment/legislative-disruption.md" target="_blank" rel="noopener noreferrer">Legislative Disruption</a></li>
</ul>
<h3><span aria-hidden="true">⚖️</span> ${riskLabel}</h3>
<ul>
<li><a href="${analysisFileBase}/risk-scoring/risk-matrix.md" target="_blank" rel="noopener noreferrer">Risk Matrix</a></li>
<li><a href="${analysisFileBase}/risk-scoring/quantitative-swot.md" target="_blank" rel="noopener noreferrer">Quantitative SWOT</a></li>
<li><a href="${analysisFileBase}/risk-scoring/political-capital-risk.md" target="_blank" rel="noopener noreferrer">Political Capital Risk</a></li>
<li><a href="${analysisFileBase}/risk-scoring/legislative-velocity-risk.md" target="_blank" rel="noopener noreferrer">Legislative Velocity Risk</a></li>
<li><a href="${analysisFileBase}/risk-scoring/agent-risk-workflow.md" target="_blank" rel="noopener noreferrer">Agent Risk Workflow</a></li>
</ul>
<h3><span aria-hidden="true">🔍</span> ${deepLabel}</h3>
<ul>
<li><a href="${analysisFileBase}/existing/deep-analysis.md" target="_blank" rel="noopener noreferrer">Deep Analysis</a></li>
<li><a href="${analysisFileBase}/existing/stakeholder-analysis.md" target="_blank" rel="noopener noreferrer">Stakeholder Analysis</a></li>
<li><a href="${analysisFileBase}/existing/coalition-analysis.md" target="_blank" rel="noopener noreferrer">Coalition Analysis</a></li>
<li><a href="${analysisFileBase}/existing/voting-patterns.md" target="_blank" rel="noopener noreferrer">Voting Patterns</a></li>
<li><a href="${analysisFileBase}/existing/cross-session-intelligence.md" target="_blank" rel="noopener noreferrer">Cross-Session Intelligence</a></li>
</ul>
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These Analysis & Transparency links are constructed as analysis/{date}/{slug}. With suffix-based deduplication in the analysis stage (e.g. analysis/{date}/{slug}-2), articles generated on repeated runs will link to the wrong (earlier) analysis artifacts. Consider passing a run-specific analysis directory name into the template so it can link to the resolved analysis output path for that run.

Copilot uses AI. Check for mistakes.
Comment on lines +443 to +455
<li><a href="${analysisFileBase}/risk-scoring/risk-matrix.md" target="_blank" rel="noopener noreferrer">Risk Matrix</a></li>
<li><a href="${analysisFileBase}/risk-scoring/quantitative-swot.md" target="_blank" rel="noopener noreferrer">Quantitative SWOT</a></li>
<li><a href="${analysisFileBase}/risk-scoring/political-capital-risk.md" target="_blank" rel="noopener noreferrer">Political Capital Risk</a></li>
<li><a href="${analysisFileBase}/risk-scoring/legislative-velocity-risk.md" target="_blank" rel="noopener noreferrer">Legislative Velocity Risk</a></li>
<li><a href="${analysisFileBase}/risk-scoring/agent-risk-workflow.md" target="_blank" rel="noopener noreferrer">Agent Risk Workflow</a></li>
</ul>
<h3><span aria-hidden="true">🔍</span> ${deepLabel}</h3>
<ul>
<li><a href="${analysisFileBase}/existing/deep-analysis.md" target="_blank" rel="noopener noreferrer">Deep Analysis</a></li>
<li><a href="${analysisFileBase}/existing/stakeholder-analysis.md" target="_blank" rel="noopener noreferrer">Stakeholder Analysis</a></li>
<li><a href="${analysisFileBase}/existing/coalition-analysis.md" target="_blank" rel="noopener noreferrer">Coalition Analysis</a></li>
<li><a href="${analysisFileBase}/existing/voting-patterns.md" target="_blank" rel="noopener noreferrer">Voting Patterns</a></li>
<li><a href="${analysisFileBase}/existing/cross-session-intelligence.md" target="_blank" rel="noopener noreferrer">Cross-Session Intelligence</a></li>
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new per-file analysis links use English-only link text (e.g. “Significance Assessment”, “Actor Mapping”) regardless of the article language (lang). Since surrounding headings are localized and the HTML document sets lang, consider localizing these link titles (or reusing existing localized labels) to avoid mixed-language navigation for screen readers and non-English readers.

Suggested change
<li><a href="${analysisFileBase}/risk-scoring/risk-matrix.md" target="_blank" rel="noopener noreferrer">Risk Matrix</a></li>
<li><a href="${analysisFileBase}/risk-scoring/quantitative-swot.md" target="_blank" rel="noopener noreferrer">Quantitative SWOT</a></li>
<li><a href="${analysisFileBase}/risk-scoring/political-capital-risk.md" target="_blank" rel="noopener noreferrer">Political Capital Risk</a></li>
<li><a href="${analysisFileBase}/risk-scoring/legislative-velocity-risk.md" target="_blank" rel="noopener noreferrer">Legislative Velocity Risk</a></li>
<li><a href="${analysisFileBase}/risk-scoring/agent-risk-workflow.md" target="_blank" rel="noopener noreferrer">Agent Risk Workflow</a></li>
</ul>
<h3><span aria-hidden="true">🔍</span> ${deepLabel}</h3>
<ul>
<li><a href="${analysisFileBase}/existing/deep-analysis.md" target="_blank" rel="noopener noreferrer">Deep Analysis</a></li>
<li><a href="${analysisFileBase}/existing/stakeholder-analysis.md" target="_blank" rel="noopener noreferrer">Stakeholder Analysis</a></li>
<li><a href="${analysisFileBase}/existing/coalition-analysis.md" target="_blank" rel="noopener noreferrer">Coalition Analysis</a></li>
<li><a href="${analysisFileBase}/existing/voting-patterns.md" target="_blank" rel="noopener noreferrer">Voting Patterns</a></li>
<li><a href="${analysisFileBase}/existing/cross-session-intelligence.md" target="_blank" rel="noopener noreferrer">Cross-Session Intelligence</a></li>
<li><a href="${analysisFileBase}/risk-scoring/risk-matrix.md" target="_blank" rel="noopener noreferrer">${getLocalizedString(lang, { en: 'Risk Matrix', sv: 'Riskmatris', da: 'Risikomatrix', no: 'Risikomatrise', fi: 'Riskimatriisi', de: 'Risikomatrix', fr: 'Matrice des risques', es: 'Matriz de riesgos', nl: 'Risicomatrix', ar: 'مصفوفة المخاطر', he: 'מטריצת סיכונים', ja: 'リスクマトリクス', ko: '위험 매트릭스', zh: '风险矩阵' })}</a></li>
<li><a href="${analysisFileBase}/risk-scoring/quantitative-swot.md" target="_blank" rel="noopener noreferrer">${getLocalizedString(lang, { en: 'Quantitative SWOT', sv: 'Kvantitativ SWOT', da: 'Kvantitativ SWOT', no: 'Kvantitativ SWOT', fi: 'Kvantitatiivinen SWOT', de: 'Quantitative SWOT', fr: 'SWOT quantitative', es: 'SWOT cuantitativo', nl: 'Kwantitatieve SWOT', ar: 'تحليل SWOT كمي', he: 'ניתוח SWOT כמותי', ja: '定量的SWOT', ko: '정량적 SWOT', zh: '定量 SWOT 分析' })}</a></li>
<li><a href="${analysisFileBase}/risk-scoring/political-capital-risk.md" target="_blank" rel="noopener noreferrer">${getLocalizedString(lang, { en: 'Political Capital Risk', sv: 'Risk för politiskt kapital', da: 'Risiko for politisk kapital', no: 'Risiko for politisk kapital', fi: 'Poliittisen pääoman riski', de: 'Risiko politischen Kapitals', fr: 'Risque de capital politique', es: 'Riesgo de capital político', nl: 'Risico op politiek kapitaal', ar: 'مخاطر رأس المال السياسي', he: 'סיכון הון פוליטי', ja: '政治資本リスク', ko: '정치적 자본 위험', zh: '政治资本风险' })}</a></li>
<li><a href="${analysisFileBase}/risk-scoring/legislative-velocity-risk.md" target="_blank" rel="noopener noreferrer">${getLocalizedString(lang, { en: 'Legislative Velocity Risk', sv: 'Risk för lagstiftningstakt', da: 'Risiko for lovgivningshastighed', no: 'Risiko for lovgivningstempo', fi: 'Lainsäädäntönopeuden riski', de: 'Risiko der Gesetzgebungsgeschwindigkeit', fr: 'Risque de vitesse législative', es: 'Riesgo de velocidad legislativa', nl: 'Risico op wetgevingssnelheid', ar: 'مخاطر وتيرة التشريع', he: 'סיכון קצב חקיקה', ja: '立法速度リスク', ko: '입법 속도 위험', zh: '立法速度风险' })}</a></li>
<li><a href="${analysisFileBase}/risk-scoring/agent-risk-workflow.md" target="_blank" rel="noopener noreferrer">${getLocalizedString(lang, { en: 'Agent Risk Workflow', sv: 'Agentarbetsflöde för risk', da: 'Agentarbejdsgang for risiko', no: 'Agentarbeidsflyt for risiko', fi: 'Agentin riskityönkulku', de: 'Agenten-Risiko-Workflow', fr: 'Flux de travail de risque de l’agent', es: 'Flujo de trabajo de riesgo del agente', nl: 'Agent-risicowerkstroom', ar: 'سير عمل مخاطر الوكيل', he: 'תהליך עבודה לסיכוני סוכן', ja: 'エージェントリスクワークフロー', ko: '에이전트 위험 워크플로', zh: '代理风险工作流' })}</a></li>
</ul>
<h3><span aria-hidden="true">🔍</span> ${deepLabel}</h3>
<ul>
<li><a href="${analysisFileBase}/existing/deep-analysis.md" target="_blank" rel="noopener noreferrer">${getLocalizedString(lang, { en: 'Deep Analysis', sv: 'Fördjupad analys', da: 'Dybdeanalyse', no: 'Dybdanalyse', fi: 'Syväanalyysi', de: 'Tiefenanalyse', fr: 'Analyse approfondie', es: 'Análisis en profundidad', nl: 'Diepgaande analyse', ar: 'تحليل معمق', he: 'ניתוח מעמיק', ja: '詳細分析', ko: '심층 분석', zh: '深度分析' })}</a></li>
<li><a href="${analysisFileBase}/existing/stakeholder-analysis.md" target="_blank" rel="noopener noreferrer">${getLocalizedString(lang, { en: 'Stakeholder Analysis', sv: 'Intressentanalys', da: 'Interessentanalyse', no: 'Interessentanalyse', fi: 'Sidosryhmäanalyysi', de: 'Stakeholder-Analyse', fr: 'Analyse des parties prenantes', es: 'Análisis de partes interesadas', nl: 'Stakeholderanalyse', ar: 'تحليل أصحاب المصلحة', he: 'ניתוח בעלי עניין', ja: 'ステークホルダー分析', ko: '이해관계자 분석', zh: '利益相关者分析' })}</a></li>
<li><a href="${analysisFileBase}/existing/coalition-analysis.md" target="_blank" rel="noopener noreferrer">${getLocalizedString(lang, { en: 'Coalition Analysis', sv: 'Koalitionsanalys', da: 'Koalitionsanalyse', no: 'Koalisjonsanalyse', fi: 'Koalitioanalyysi', de: 'Koalitionsanalyse', fr: 'Analyse des coalitions', es: 'Análisis de coaliciones', nl: 'Coalitieanalyse', ar: 'تحليل الائتلافات', he: 'ניתוח קואליציות', ja: '連立分析', ko: '연합 분석', zh: '联盟分析' })}</a></li>
<li><a href="${analysisFileBase}/existing/voting-patterns.md" target="_blank" rel="noopener noreferrer">${getLocalizedString(lang, { en: 'Voting Patterns', sv: 'Röstmönster', da: 'Stemmemønstre', no: 'Stemmemønstre', fi: 'Äänestysmallit', de: 'Abstimmungsmuster', fr: 'Schémas de vote', es: 'Patrones de votación', nl: 'Stempatronen', ar: 'أنماط التصويت', he: 'דפוסי הצבעה', ja: '投票パターン', ko: '투표 패턴', zh: '投票模式' })}</a></li>
<li><a href="${analysisFileBase}/existing/cross-session-intelligence.md" target="_blank" rel="noopener noreferrer">${getLocalizedString(lang, { en: 'Cross-Session Intelligence', sv: 'Tvärsessionsintelligens', da: 'Tværsessionsintelligens', no: 'Tverrsesjonsintelligens', fi: 'Istuntojen välinen tiedusteluanalyysi', de: 'Sitzungsübergreifende Analyse', fr: 'Analyse intersessions', es: 'Inteligencia entre sesiones', nl: 'Sessie-overstijgende analyse', ar: 'تحليل عبر الجلسات', he: 'מודיעין בין-מושבי', ja: '会期間インテリジェンス', ko: '세션 간 인텔리전스', zh: '跨会期情报分析' })}</a></li>

Copilot uses AI. Check for mistakes.
@pethers
Copy link
Copy Markdown
Member

pethers commented Apr 2, 2026

@copilot apply changes based on the comments in this thread and investigate failing checks
Test and Report / Validate Code (pull_request)
Test and Report / Validate Code (pull_request)Failing after 55s

Copilot stopped work on behalf of pethers due to an error April 2, 2026 23:39
Copilot stopped work on behalf of pethers due to an error April 2, 2026 23:45
@pethers
Copy link
Copy Markdown
Member

pethers commented Apr 2, 2026

@copilot apply changes based on the comments in this thread

Copilot stopped work on behalf of pethers due to an error April 2, 2026 23:52
Copilot stopped work on behalf of pethers due to an error April 3, 2026 00:02
@pethers
Copy link
Copy Markdown
Member

pethers commented Apr 3, 2026

@copilot apply changes based on the comments in this thread

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 3, 2026

@pethers Unfortunately I hit an unexpected error while processing your comment. I've automatically reported this to GitHub.

You can ask me to try again later by mentioning me in a new comment.

If you want to contact GitHub about this error, please mention the following identifier so they can better serve you: 5d81127f-c16f-4c3c-a2f9-f111695b5492

Sorry for the inconvenience!

…ming, JSDoc, guide example

Agent-Logs-Url: https://github.com/Hack23/euparliamentmonitor/sessions/9d3f226a-0aa3-4237-a29a-62ca2651ab04

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 3, 2026

@copilot apply changes based on the comments in this thread

All 5 review comments addressed in commit 6b57440:

  1. slug in templategenerateSingleLanguageArticle now extracts the typeSlug from the file slug via slug.replace(/^\d{4}-\d{2}-\d{2}-/, ''), so canonical/og:url/language-switcher links include the dedup suffix.

  2. Per-strategy dedup suffix — Instead of typeSlug = analysisDir ?? strategy.type, we now extract just the numeric suffix from analysisDir (/-(\d+)$/) and append it to each strategy.type independently. Multi-strategy runs produce distinct slugs (e.g. breaking-2 and week-ahead-2).

  3. Atomic dir claimingresolveUniqueAnalysisDir uses mkdirSync(candidate, {recursive:false}) with EEXIST retry for suffixed candidates. Base dir still uses manifest.json check (needed for incremental runs where the dir exists but analysis is in progress).

  4. JSDoc — Updated to accurately describe slug-level dedup (no file-level dedup).

  5. Guide example — Fixed to breaking-2-en.html (suffix in slug, before lang code).

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated 1 comment.

Comment on lines +245 to +250
// If the analysis directory was deduplicated (e.g. "breaking-2"), extract
// just the numeric suffix and apply it to the current strategy's type.
// This keeps per-strategy slugs distinct when multiple article types are
// generated in a single run (e.g. "breaking-2" and "week-ahead-2").
const suffixMatch = analysisDir?.match(/-(\d+)$/);
const dedupSuffix = suffixMatch ? `-${suffixMatch[1]}` : '';
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dedupSuffix extraction only supports numeric -(\d+) suffixes. If resolveUniqueAnalysisDir() falls back to the UUID suffix (or any future non-numeric suffix), article slugs won’t include the suffix, so repeated runs can still overwrite news/{date}-{type}-{lang}.html and the article’s canonical/og:url won’t line up with the deduplicated analysis directory.

Consider propagating the full suffix from the resolved analysis directory (numeric or UUID) and appending that to each strategy.type, instead of restricting to digits only.

Suggested change
// If the analysis directory was deduplicated (e.g. "breaking-2"), extract
// just the numeric suffix and apply it to the current strategy's type.
// This keeps per-strategy slugs distinct when multiple article types are
// generated in a single run (e.g. "breaking-2" and "week-ahead-2").
const suffixMatch = analysisDir?.match(/-(\d+)$/);
const dedupSuffix = suffixMatch ? `-${suffixMatch[1]}` : '';
// If the analysis directory was deduplicated (e.g. "breaking-2" or a
// UUID-based fallback such as "breaking-550e8400-e29b-41d4-a716-446655440000"),
// propagate the full suffix for the current strategy type into the slug.
// This keeps per-strategy slugs distinct and aligned with the resolved
// analysis directory regardless of suffix format.
const dedupPrefix = `${strategy.type}-`;
const dedupSuffix =
analysisDir !== undefined && analysisDir.startsWith(dedupPrefix)
? analysisDir.slice(strategy.type.length)
: '';

Copilot uses AI. Check for mistakes.
@pethers
Copy link
Copy Markdown
Member

pethers commented Apr 3, 2026

@copilot apply changes based on the comments in this thread and analyse Code scanning results / CodeQL
Code scanning results / CodeQLFailing after 2s — 14 new alerts including 14 high severity security vulnerabilities

Copilot AI and others added 2 commits April 3, 2026 03:20
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 3, 2026

@copilot apply changes based on the comments in this thread and analyse [Code scanning results / ...

All changes applied in commits 302c7a0 and 86c2ccc:

  1. Full suffix support — Replaced numeric-only regex (/-(\d+)$/) with prefix-based matching via new deriveTypeSlug() helper. Both numeric (breaking-2) and UUID-based (breaking-a1b2c3d4) dedup suffixes now propagate correctly to article slugs.

  2. Cognitive complexity fix — Extracted deriveTypeSlug() helper to bring generateArticleForStrategy back under the sonarjs/cognitive-complexity limit (≤15). 0 lint errors, 2713 tests pass.

  3. CodeQL analysis — All 14 high-severity alerts are pre-existing on main (created 2026-03-16 to 2026-04-01), none introduced by this PR:

    • 12× js/polynomial-redos in content-validator.ts, article-quality-scorer.ts, content-metadata.ts, section-builders.ts, article-template.ts
    • Token-Permissions in workflow files
    • js/xss-through-dom in test coverage reports
    • js/incomplete-hostname-regexp in article-template.ts

@copilot apply changes based on the comments in this thread

Addressed in commits 302c7a0 and 86c2ccc. See the later comment for full details — deriveTypeSlug() helper now handles both numeric and UUID suffixes via prefix-based matching, fixing the review comment. CodeQL alerts are all pre-existing on main.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 24 out of 24 changed files in this pull request and generated 5 comments.

Comment on lines 2729 to 2736
// resolveUniqueAnalysisDir appends a numeric suffix (-2, -3, …) when
// a prior completed run (with manifest.json) already occupies the path,
// preventing repeated workflow runs from overwriting previous analysis.
const preferredDir = articleTypeSlug
? path.resolve(outputDir, date, articleTypeSlug)
: path.resolve(outputDir, date);
const dateOutputDir = resolveUniqueAnalysisDir(preferredDir);

Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

runAnalysisStage() calls resolveUniqueAnalysisDir(preferredDir) before validating hasSubstantiveData(fetchedData). Since resolveUniqueAnalysisDir may create a suffixed directory immediately, a run that later aborts (e.g. requireData=true with empty/invalid data, or another preflight error) can leave behind an empty claimed directory (e.g. .../breaking-2) that forces subsequent runs to skip to -3, etc. Consider moving the substantive-data validation earlier (before claiming/creating any directories), or cleaning up the claimed directory on early failure.

Copilot uses AI. Check for mistakes.
Comment on lines +113 to +118
function deriveTypeSlug(strategyType: string, analysisDir?: string): string {
const prefix = `${strategyType}-`;
if (analysisDir !== undefined && analysisDir.startsWith(prefix)) {
return analysisDir;
}
return strategyType;
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deriveTypeSlug() returns the full analysisDir when it starts with ${strategyType}-, which breaks when analysisDir is a multi-type slug produced by deriveArticleTypeSlug (e.g. breaking-week-ahead-2): the “breaking” article would incorrectly get typeSlug breaking-week-ahead-2. It also does not implement the PR’s stated behavior of extracting the dedup suffix (e.g. -2) and applying it per strategy type. Consider parsing the suffix from analysisDir (numeric/UUID) and appending it to each strategyType, instead of substituting analysisDir wholesale.

Copilot uses AI. Check for mistakes.
Comment on lines 506 to +549
@@ -534,7 +539,14 @@ async function main(): Promise<void> {
}

results.push(
await generateArticleForStrategy(strategy, client, languages, outputOptions, stats)
await generateArticleForStrategy(
strategy,
client,
languages,
outputOptions,
stats,
analysisDir
)
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

news-enhanced.ts passes a single analysisDir (basename of analysisCtx.outputDir) to generateArticleForStrategy() for every requested article type. When multiple articleTypes are generated in one invocation, the analysis output directory slug is derived from all types (deriveArticleTypeSlug), so analysisDir will not match each individual strategy’s analysis folder. This will make Analysis & Transparency links (and potentially the deduped type slug logic) point to the wrong directory for some articles. A safer approach is to compute a per-strategy analysisDir (or just a suffix) and pass that per strategy instead of reusing the run-level basename for all strategies.

Copilot uses AI. Check for mistakes.
Comment on lines +161 to +166
export function resolveUniqueAnalysisDir(baseDir: string): string {
// If the directory doesn't exist yet or has no manifest from a prior
// completed run, use it as-is. A directory without manifest.json is
// considered available (not yet finished by any run).
if (!fs.existsSync(path.join(baseDir, 'manifest.json'))) {
return baseDir;
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR description says “Any existing directory … is treated as occupied” for slug-level dedup, but resolveUniqueAnalysisDir() currently treats a directory as occupied only when manifest.json exists (and even comments that a directory without manifest is considered available). If the intended contract is “any existing dir is occupied”, the implementation and JSDoc/comment here should be updated to match (or the PR description adjusted).

Copilot uses AI. Check for mistakes.
Comment on lines +100 to +112
/**
* Extract the deduplication suffix from a resolved analysis directory name
* and apply it to the given strategy type.
*
* If `analysisDir` is `"breaking-2"` and `strategyType` is `"breaking"`,
* returns `"breaking-2"`. For UUID-based fallbacks like `"breaking-a1b2c3d4"`,
* returns `"breaking-a1b2c3d4"`. When no suffix applies, returns the
* `strategyType` unchanged.
*
* @param strategyType - Base article type (e.g. `"breaking"`)
* @param analysisDir - Resolved analysis directory basename, if any
* @returns The type slug with any dedup suffix appended
*/
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new dedup-related behavior (deriveTypeSlug + propagation of analysisDir into generateArticleHTML for canonical URLs / transparency links) is not covered by unit tests. There are existing generate-stage pipeline tests, but none assert that a suffixed analysis run (e.g. breaking-2) results in (a) the expected article slug/filename pattern and (b) transparency links pointing at the suffixed analysis directory. Adding focused Vitest cases for numeric and UUID suffixes (and for multi-article-type slugs) would help prevent regressions.

Copilot generated this review using guidance from repository custom instructions.
@pethers
Copy link
Copy Markdown
Member

pethers commented Apr 3, 2026

@copilot apply changes based on the comments in this thread

Copilot AI and others added 2 commits April 3, 2026 04:26
…ing, move data validation before dir claiming, add dedup unit tests

- deriveTypeSlug now takes a dedupSuffix string (e.g. "-2") instead of full analysisDir, fixing multi-type slug corruption
- news-enhanced.ts extracts dedupSuffix by comparing resolved dir basename with base slug
- resolveUniqueAnalysisDir uses manifest.json check for base dir (supports skipCompleted) + atomic mkdirSync for suffixed candidates
- analysis-stage.ts moves hasSubstantiveData check before resolveUniqueAnalysisDir to prevent orphan dirs on abort
- Added 11 focused Vitest tests for deriveTypeSlug, generateArticleForStrategy with dedup, and resolveUniqueAnalysisDir

Agent-Logs-Url: https://github.com/Hack23/euparliamentmonitor/sessions/07cc3829-1346-4465-8560-d2fabe1befc6

Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 3, 2026

@copilot apply changes based on the comments in this thread

All 5 review comments addressed in commits f06f363 and bd4c912:

  1. Data validation before dir claiminghasSubstantiveData check moved before resolveUniqueAnalysisDir in analysis-stage.ts, preventing orphan directories on early abort.

  2. Fixed deriveTypeSlug() for multi-type slugs — Now takes a dedupSuffix string (e.g. "-2") instead of full analysisDir, so multi-type analysis dirs like breaking-week-ahead-2 correctly produce per-strategy slugs breaking-2 and week-ahead-2.

  3. Per-strategy suffix in news-enhanced.ts — Computes dedupSuffix by comparing resolved dir basename with base slug, validated against /^-[\da-f]+$/i. Passes both dedupSuffix and analysisDir separately to generateArticleForStrategy.

  4. Aligned JSDoc with behaviorresolveUniqueAnalysisDir uses manifest.json check for base dir (supports skipCompleted) + atomic claimDir for suffixed candidates (race-safe).

  5. Added 11 Vitest testsderiveTypeSlug (numeric/UUID suffixes, per-strategy distinctness), generateArticleForStrategy with dedup, resolveUniqueAnalysisDir (base/suffixed/manifest/no-manifest).

CodeQL: All 14 alerts are pre-existing js/polynomial-redos in content-validator.ts and article-quality-scorer.ts — none introduced by this PR.

@pethers
Copy link
Copy Markdown
Member

pethers commented Apr 3, 2026

@copilot analyse and fix Test and Report / Validate Code (pull_request)
Test and Report / Validate Code (pull_request)Failing after 1m

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 3, 2026

@copilot apply changes based on the comments in this thread

All 5 review comments addressed in commits f06f363 and bd4c912:

  1. Data validation before dir claiminghasSubstantiveData check moved before resolveUniqueAnalysisDir in analysis-stage.ts, preventing orphan directories on early abort.

  2. Fixed deriveTypeSlug() for multi-type slugs — Now takes a dedupSuffix string (e.g. "-2") instead of full analysisDir, so multi-type analysis dirs like breaking-week-ahead-2 correctly produce per-strategy slugs breaking-2 and week-ahead-2.

  3. Per-strategy suffix in news-enhanced.ts — Computes dedupSuffix by comparing resolved dir basename with base slug, validated against /^-[\da-f]+$/i. Passes both dedupSuffix and analysisDir separately to generateArticleForStrategy.

  4. Aligned JSDoc with behaviorresolveUniqueAnalysisDir uses manifest.json check for base dir (supports skipCompleted) + atomic claimDir for suffixed candidates (race-safe).

  5. Added 11 Vitest testsderiveTypeSlug (numeric/UUID suffixes, per-strategy distinctness), generateArticleForStrategy with dedup, resolveUniqueAnalysisDir (base/suffixed/manifest/no-manifest).

@copilot analyse and fix Test and Report / Validate Code (pull_request)
Test and Repor...

CI failure was a Prettier formatting issue in src/generators/news-enhanced.ts. Fixed in commit c2f73da.

@pethers pethers merged commit 8c3893f into main Apr 3, 2026
13 of 14 checks passed
@pethers pethers deleted the copilot/analyze-daily-content-news-2026-04-02 branch April 3, 2026 05:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

content Content updates and improvements documentation Documentation updates github_actions GitHub Actions updates infrastructure CI/CD and build infrastructure needs-tests Needs test coverage news News generation and articles scripts Scripts and automation test-coverage-impact Impacts test coverage testing Testing improvements workflows GitHub workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants