Conversation
…e analysis links - Add Rules 9-12 to ai-driven-analysis-guide.md for AI-driven headlines, analysis linkage, prompt placeholders, and narrative separation - Update article template to link to all 18 individual analysis files instead of just directory-level links - Transform 5 existing analysis methods to produce AI prompt placeholders instead of bare data tables - Add AI-driven headline/description generation sections to all 4 news workflows (breaking, committee-reports, motions, propositions) - Add analysis file reference verification checklists to all workflows Agent-Logs-Url: https://github.com/Hack23/euparliamentmonitor/sessions/63626a43-654f-44ed-8da2-135dc82cb84e Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Agent-Logs-Url: https://github.com/Hack23/euparliamentmonitor/sessions/63626a43-654f-44ed-8da2-135dc82cb84e Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
…ticles on repeated runs - Add resolveUniqueAnalysisDir() to file-utils.ts: appends -2, -3, etc. when manifest.json exists - Add resolveUniqueFilePath() to file-utils.ts: appends -2, -3, etc. to article filenames - Update analysis-stage.ts to use resolveUniqueAnalysisDir before creating output dir - Update output-stage.ts writeArticleFile to use resolveUniqueFilePath instead of overwriting - Update ai-driven-analysis-guide.md Rule 1 to document suffix deduplication behavior - Update tests to match new non-overwrite behavior Agent-Logs-Url: https://github.com/Hack23/euparliamentmonitor/sessions/ef87e601-17be-49e0-81d8-5c34cdc34cd5 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Agent-Logs-Url: https://github.com/Hack23/euparliamentmonitor/sessions/ef87e601-17be-49e0-81d8-5c34cdc34cd5 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Prevents repeated same-day workflow runs from overwriting prior analysis outputs and generated news articles by introducing suffix-based deduplication, and expands AI analysis guidance/prompts plus article transparency linking to individual analysis artifacts.
Changes:
- Added utilities to resolve unique analysis directories and unique article file paths via numeric suffixing (
-2,-3, …). - Updated pipeline stages to use the new deduplication utilities and adjusted unit tests accordingly.
- Expanded AI analysis templates/prompts and updated article templates + workflows to require AI-driven headlines/descriptions and link to individual analysis files.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
src/utils/file-utils.ts |
Adds resolveUniqueAnalysisDir() and resolveUniqueFilePath() utilities. |
src/generators/pipeline/analysis-stage.ts |
Uses resolveUniqueAnalysisDir() to avoid overwriting prior analysis runs. |
src/generators/pipeline/output-stage.ts |
Uses resolveUniqueFilePath() to avoid overwriting existing articles. |
src/templates/article-template.ts |
Expands Analysis & Transparency section to link to individual analysis .md files. |
scripts/utils/file-utils.js |
Compiled JS equivalent of new file util functions. |
scripts/generators/pipeline/analysis-stage.js |
Compiled JS equivalent of analysis-stage deduplication changes. |
scripts/generators/pipeline/output-stage.js |
Compiled JS equivalent of output-stage deduplication changes. |
scripts/templates/article-template.js |
Compiled JS equivalent of expanded transparency links. |
test/unit/pipeline-stages.test.js |
Updates expectations: no overwrite; writes suffixed article filenames. |
test/unit/analysis-stage.test.js |
Updates expectations: second run writes to suffixed analysis directory / adjusted assertions. |
analysis/methodologies/ai-driven-analysis-guide.md |
Updates guide to v3.0 including dedup + headline/linkage rules. |
.github/workflows/news-breaking.md |
Adds mandatory AI-driven headline/description + analysis link verification guidance. |
.github/workflows/news-committee-reports.md |
Adds mandatory AI-driven headline/description + analysis link verification guidance. |
.github/workflows/news-motions.md |
Adds mandatory AI-driven headline/description + analysis link verification guidance. |
.github/workflows/news-propositions.md |
Adds mandatory AI-driven headline/description + analysis link verification guidance. |
Comments suppressed due to low confidence (1)
scripts/generators/pipeline/output-stage.js:71
writeArticleFile()may write to a suffixed filename viaresolveUniqueFilePath(), but it still returns only a boolean.writeSingleArticle()then pushes the originalfilenameintostats.articles, so the generation metadata can reference files that were never written (when deduplication occurs). Consider returning the actual written filename/path fromwriteArticleFile()and using it when updating stats.
// When not in skip mode: resolve a unique path to avoid overwriting
// existing articles from prior workflow runs on the same date.
const uniquePath = resolveUniqueFilePath(filepath);
atomicWrite(uniquePath, html);
const writtenName = path.basename(uniquePath);
if (writtenName !== filename) {
console.log(` ✅ Wrote: ${writtenName} (unique — original ${filename} already existed)`);
}
else {
console.log(` ✅ Wrote: ${filename}`);
}
return true;
}
/**
* Write a language-specific article file and update the generation stats.
*
* @param html - Full HTML content to write
* @param slug - Article slug (e.g. `"2025-01-15-week-ahead"`)
* @param lang - Language code suffix (e.g. `"en"`)
* @param options - Output flags and directory path
* @param stats - Mutable stats object to increment counters on
* @returns `true` when the file was actually written
*/
export function writeSingleArticle(html, slug, lang, options, stats) {
const filename = `${slug}-${lang}.html`;
const written = writeArticleFile(html, filename, options);
if (written) {
stats.generated += 1;
stats.articles.push(filename);
}
| // When not in skip mode: resolve a unique path to avoid overwriting | ||
| // existing articles from prior workflow runs on the same date. | ||
| const uniquePath = resolveUniqueFilePath(filepath); | ||
| atomicWrite(uniquePath, html); | ||
| const writtenName = path.basename(uniquePath); | ||
| if (writtenName !== filename) { | ||
| console.log(` ✅ Wrote: ${writtenName} (unique — original ${filename} already existed)`); | ||
| } else { | ||
| console.log(` ✅ Wrote: ${filename}`); | ||
| } |
There was a problem hiding this comment.
writeArticleFile now writes to a potentially suffixed filename via resolveUniqueFilePath(), but it still only returns a boolean. Callers like writeSingleArticle() (same module) record filename into stats.articles, so a deduplicated write will cause metadata/stats to reference a file that was never created. Consider returning the actual written filename/path (or exposing it via an out param) and updating stats/logging to use that value.
| // resolveUniqueAnalysisDir appends a numeric suffix (-2, -3, …) when | ||
| // a prior completed run (with manifest.json) already occupies the path, | ||
| // preventing repeated workflow runs from overwriting previous analysis. | ||
| const preferredDir = articleTypeSlug | ||
| ? path.resolve(outputDir, date, articleTypeSlug) | ||
| : path.resolve(outputDir, date); | ||
| const dateOutputDir = resolveUniqueAnalysisDir(preferredDir); |
There was a problem hiding this comment.
resolveUniqueAnalysisDir(preferredDir) can produce a suffixed directory like analysis/{date}/{slug}-2, but the article transparency links are built from {date} + {slug} only. On repeated same-day runs, newly generated articles will link to the original analysis directory instead of the run-specific suffixed one, breaking provenance. Consider passing the resolved analysis directory name (from AnalysisContext.outputDir or the manifest) into generateArticleHTML() / renderAnalysisTransparencySection() so links match the actual output path.
| const repoBase = 'https://github.com/Hack23/euparliamentmonitor/blob/main'; | ||
| const treeDirBase = 'https://github.com/Hack23/euparliamentmonitor/tree/main'; | ||
| const analysisDir = `${treeDirBase}/analysis/${safeDate}/${safeSlug}`; | ||
| const analysisFileBase = `${repoBase}/analysis/${safeDate}/${safeSlug}`; | ||
| const methodologyDir = `${repoBase}/analysis/methodologies`; | ||
|
|
||
| return ` | ||
| <section class="analysis-transparency" aria-label="${heading}"> | ||
| <h2 id="analysis-transparency-heading">${heading}</h2> | ||
| <p>${disclosure}</p> | ||
| <nav class="analysis-links" aria-labelledby="analysis-transparency-heading"> | ||
| <h3><span aria-hidden="true">📊</span> ${analysisSummaryLabel}</h3> | ||
| <ul> | ||
| <li><a href="${analysisDir}" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">📊</span> ${analysisSummaryLabel}</a></li> | ||
| <li><a href="${analysisDir}/classification" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">🏷️</span> ${classificationLabel}</a></li> | ||
| <li><a href="${analysisDir}/threat-assessment" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">🛡️</span> ${threatLabel}</a></li> | ||
| <li><a href="${analysisDir}/risk-scoring" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">⚖️</span> ${riskLabel}</a></li> | ||
| <li><a href="${analysisDir}/existing" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">🔍</span> ${deepLabel}</a></li> | ||
| <li><a href="${analysisDir}" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">📁</span> ${analysisSummaryLabel}</a></li> | ||
| </ul> | ||
| <h3><span aria-hidden="true">🏷️</span> ${classificationLabel}</h3> | ||
| <ul> | ||
| <li><a href="${analysisFileBase}/classification/significance-assessment.md" target="_blank" rel="noopener noreferrer">Significance Assessment</a></li> | ||
| <li><a href="${analysisFileBase}/classification/actor-mapping.md" target="_blank" rel="noopener noreferrer">Actor Mapping</a></li> | ||
| <li><a href="${analysisFileBase}/classification/forces-analysis.md" target="_blank" rel="noopener noreferrer">Forces Analysis</a></li> | ||
| <li><a href="${analysisFileBase}/classification/impact-matrix.md" target="_blank" rel="noopener noreferrer">Impact Matrix</a></li> | ||
| </ul> | ||
| <h3><span aria-hidden="true">🛡️</span> ${threatLabel}</h3> | ||
| <ul> | ||
| <li><a href="${analysisFileBase}/threat-assessment/political-threat-landscape.md" target="_blank" rel="noopener noreferrer">Political Threat Landscape</a></li> | ||
| <li><a href="${analysisFileBase}/threat-assessment/actor-threat-profiles.md" target="_blank" rel="noopener noreferrer">Actor Threat Profiles</a></li> | ||
| <li><a href="${analysisFileBase}/threat-assessment/consequence-trees.md" target="_blank" rel="noopener noreferrer">Consequence Trees</a></li> | ||
| <li><a href="${analysisFileBase}/threat-assessment/legislative-disruption.md" target="_blank" rel="noopener noreferrer">Legislative Disruption</a></li> | ||
| </ul> | ||
| <h3><span aria-hidden="true">⚖️</span> ${riskLabel}</h3> | ||
| <ul> | ||
| <li><a href="${analysisFileBase}/risk-scoring/risk-matrix.md" target="_blank" rel="noopener noreferrer">Risk Matrix</a></li> | ||
| <li><a href="${analysisFileBase}/risk-scoring/quantitative-swot.md" target="_blank" rel="noopener noreferrer">Quantitative SWOT</a></li> | ||
| <li><a href="${analysisFileBase}/risk-scoring/political-capital-risk.md" target="_blank" rel="noopener noreferrer">Political Capital Risk</a></li> | ||
| <li><a href="${analysisFileBase}/risk-scoring/legislative-velocity-risk.md" target="_blank" rel="noopener noreferrer">Legislative Velocity Risk</a></li> | ||
| <li><a href="${analysisFileBase}/risk-scoring/agent-risk-workflow.md" target="_blank" rel="noopener noreferrer">Agent Risk Workflow</a></li> | ||
| </ul> | ||
| <h3><span aria-hidden="true">🔍</span> ${deepLabel}</h3> | ||
| <ul> | ||
| <li><a href="${analysisFileBase}/existing/deep-analysis.md" target="_blank" rel="noopener noreferrer">Deep Analysis</a></li> | ||
| <li><a href="${analysisFileBase}/existing/stakeholder-analysis.md" target="_blank" rel="noopener noreferrer">Stakeholder Analysis</a></li> | ||
| <li><a href="${analysisFileBase}/existing/coalition-analysis.md" target="_blank" rel="noopener noreferrer">Coalition Analysis</a></li> | ||
| <li><a href="${analysisFileBase}/existing/voting-patterns.md" target="_blank" rel="noopener noreferrer">Voting Patterns</a></li> | ||
| <li><a href="${analysisFileBase}/existing/cross-session-intelligence.md" target="_blank" rel="noopener noreferrer">Cross-Session Intelligence</a></li> | ||
| </ul> |
There was a problem hiding this comment.
This section hard-codes analysis links to analysis/{date}/{slug}. With suffix deduplication in the analysis stage (e.g. analysis/{date}/{slug}-2), repeated workflow runs will produce articles whose Analysis & Transparency links point to the wrong (earlier) analysis artifacts. The template likely needs an additional parameter (e.g. analysisSlug / analysisDirName) so it can link to the resolved analysis output directory for the specific run.
src/templates/article-template.ts
Outdated
| <h3><span aria-hidden="true">🏷️</span> ${classificationLabel}</h3> | ||
| <ul> | ||
| <li><a href="${analysisFileBase}/classification/significance-assessment.md" target="_blank" rel="noopener noreferrer">Significance Assessment</a></li> | ||
| <li><a href="${analysisFileBase}/classification/actor-mapping.md" target="_blank" rel="noopener noreferrer">Actor Mapping</a></li> | ||
| <li><a href="${analysisFileBase}/classification/forces-analysis.md" target="_blank" rel="noopener noreferrer">Forces Analysis</a></li> | ||
| <li><a href="${analysisFileBase}/classification/impact-matrix.md" target="_blank" rel="noopener noreferrer">Impact Matrix</a></li> | ||
| </ul> | ||
| <h3><span aria-hidden="true">🛡️</span> ${threatLabel}</h3> | ||
| <ul> | ||
| <li><a href="${analysisFileBase}/threat-assessment/political-threat-landscape.md" target="_blank" rel="noopener noreferrer">Political Threat Landscape</a></li> | ||
| <li><a href="${analysisFileBase}/threat-assessment/actor-threat-profiles.md" target="_blank" rel="noopener noreferrer">Actor Threat Profiles</a></li> | ||
| <li><a href="${analysisFileBase}/threat-assessment/consequence-trees.md" target="_blank" rel="noopener noreferrer">Consequence Trees</a></li> | ||
| <li><a href="${analysisFileBase}/threat-assessment/legislative-disruption.md" target="_blank" rel="noopener noreferrer">Legislative Disruption</a></li> | ||
| </ul> | ||
| <h3><span aria-hidden="true">⚖️</span> ${riskLabel}</h3> | ||
| <ul> | ||
| <li><a href="${analysisFileBase}/risk-scoring/risk-matrix.md" target="_blank" rel="noopener noreferrer">Risk Matrix</a></li> | ||
| <li><a href="${analysisFileBase}/risk-scoring/quantitative-swot.md" target="_blank" rel="noopener noreferrer">Quantitative SWOT</a></li> | ||
| <li><a href="${analysisFileBase}/risk-scoring/political-capital-risk.md" target="_blank" rel="noopener noreferrer">Political Capital Risk</a></li> | ||
| <li><a href="${analysisFileBase}/risk-scoring/legislative-velocity-risk.md" target="_blank" rel="noopener noreferrer">Legislative Velocity Risk</a></li> | ||
| <li><a href="${analysisFileBase}/risk-scoring/agent-risk-workflow.md" target="_blank" rel="noopener noreferrer">Agent Risk Workflow</a></li> | ||
| </ul> | ||
| <h3><span aria-hidden="true">🔍</span> ${deepLabel}</h3> | ||
| <ul> | ||
| <li><a href="${analysisFileBase}/existing/deep-analysis.md" target="_blank" rel="noopener noreferrer">Deep Analysis</a></li> | ||
| <li><a href="${analysisFileBase}/existing/stakeholder-analysis.md" target="_blank" rel="noopener noreferrer">Stakeholder Analysis</a></li> | ||
| <li><a href="${analysisFileBase}/existing/coalition-analysis.md" target="_blank" rel="noopener noreferrer">Coalition Analysis</a></li> | ||
| <li><a href="${analysisFileBase}/existing/voting-patterns.md" target="_blank" rel="noopener noreferrer">Voting Patterns</a></li> | ||
| <li><a href="${analysisFileBase}/existing/cross-session-intelligence.md" target="_blank" rel="noopener noreferrer">Cross-Session Intelligence</a></li> |
There was a problem hiding this comment.
The new per-file analysis links use English-only link text (e.g. “Significance Assessment”, “Actor Mapping”) even when generating non-English articles (<article lang="...">). Since the surrounding headings/labels are localized, this is a regression for multilingual UX and screen readers. Consider localizing these link titles (or at least reusing existing localized labels where they exist) so the section language matches the document language.
| // When articleTypeSlug is provided, scope output to a per-article-type | ||
| // subdirectory so concurrent workflows on the same date never collide. | ||
| const dateOutputDir = articleTypeSlug | ||
| // resolveUniqueAnalysisDir appends a numeric suffix (-2, -3, …) when | ||
| // a prior completed run (with manifest.json) already occupies the path, | ||
| // preventing repeated workflow runs from overwriting previous analysis. | ||
| const preferredDir = articleTypeSlug | ||
| ? path.resolve(outputDir, date, articleTypeSlug) | ||
| : path.resolve(outputDir, date); | ||
| const dateOutputDir = resolveUniqueAnalysisDir(preferredDir); |
There was a problem hiding this comment.
resolveUniqueAnalysisDir(preferredDir) can return a suffixed path (e.g. analysis/{date}/{slug}-2), but article generation and linking logic elsewhere appears to still key off {date} + {slug}. That means repeated same-day runs can generate new analysis in a suffixed directory while produced articles still point to / consume the original directory, breaking run provenance. Consider propagating the resolved output directory (or its basename) through the pipeline so consumers link to the correct run folder.
| const repoBase = 'https://github.com/Hack23/euparliamentmonitor/blob/main'; | ||
| const treeDirBase = 'https://github.com/Hack23/euparliamentmonitor/tree/main'; | ||
| const analysisDir = `${treeDirBase}/analysis/${safeDate}/${safeSlug}`; | ||
| const analysisFileBase = `${repoBase}/analysis/${safeDate}/${safeSlug}`; | ||
| const methodologyDir = `${repoBase}/analysis/methodologies`; | ||
| return ` | ||
| <section class="analysis-transparency" aria-label="${heading}"> | ||
| <h2 id="analysis-transparency-heading">${heading}</h2> | ||
| <p>${disclosure}</p> | ||
| <nav class="analysis-links" aria-labelledby="analysis-transparency-heading"> | ||
| <h3><span aria-hidden="true">📊</span> ${analysisSummaryLabel}</h3> | ||
| <ul> | ||
| <li><a href="${analysisDir}" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">📊</span> ${analysisSummaryLabel}</a></li> | ||
| <li><a href="${analysisDir}/classification" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">🏷️</span> ${classificationLabel}</a></li> | ||
| <li><a href="${analysisDir}/threat-assessment" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">🛡️</span> ${threatLabel}</a></li> | ||
| <li><a href="${analysisDir}/risk-scoring" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">⚖️</span> ${riskLabel}</a></li> | ||
| <li><a href="${analysisDir}/existing" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">🔍</span> ${deepLabel}</a></li> | ||
| <li><a href="${analysisDir}" target="_blank" rel="noopener noreferrer"><span aria-hidden="true">📁</span> ${analysisSummaryLabel}</a></li> | ||
| </ul> | ||
| <h3><span aria-hidden="true">🏷️</span> ${classificationLabel}</h3> | ||
| <ul> | ||
| <li><a href="${analysisFileBase}/classification/significance-assessment.md" target="_blank" rel="noopener noreferrer">Significance Assessment</a></li> | ||
| <li><a href="${analysisFileBase}/classification/actor-mapping.md" target="_blank" rel="noopener noreferrer">Actor Mapping</a></li> | ||
| <li><a href="${analysisFileBase}/classification/forces-analysis.md" target="_blank" rel="noopener noreferrer">Forces Analysis</a></li> | ||
| <li><a href="${analysisFileBase}/classification/impact-matrix.md" target="_blank" rel="noopener noreferrer">Impact Matrix</a></li> | ||
| </ul> | ||
| <h3><span aria-hidden="true">🛡️</span> ${threatLabel}</h3> | ||
| <ul> | ||
| <li><a href="${analysisFileBase}/threat-assessment/political-threat-landscape.md" target="_blank" rel="noopener noreferrer">Political Threat Landscape</a></li> | ||
| <li><a href="${analysisFileBase}/threat-assessment/actor-threat-profiles.md" target="_blank" rel="noopener noreferrer">Actor Threat Profiles</a></li> | ||
| <li><a href="${analysisFileBase}/threat-assessment/consequence-trees.md" target="_blank" rel="noopener noreferrer">Consequence Trees</a></li> | ||
| <li><a href="${analysisFileBase}/threat-assessment/legislative-disruption.md" target="_blank" rel="noopener noreferrer">Legislative Disruption</a></li> | ||
| </ul> | ||
| <h3><span aria-hidden="true">⚖️</span> ${riskLabel}</h3> | ||
| <ul> | ||
| <li><a href="${analysisFileBase}/risk-scoring/risk-matrix.md" target="_blank" rel="noopener noreferrer">Risk Matrix</a></li> | ||
| <li><a href="${analysisFileBase}/risk-scoring/quantitative-swot.md" target="_blank" rel="noopener noreferrer">Quantitative SWOT</a></li> | ||
| <li><a href="${analysisFileBase}/risk-scoring/political-capital-risk.md" target="_blank" rel="noopener noreferrer">Political Capital Risk</a></li> | ||
| <li><a href="${analysisFileBase}/risk-scoring/legislative-velocity-risk.md" target="_blank" rel="noopener noreferrer">Legislative Velocity Risk</a></li> | ||
| <li><a href="${analysisFileBase}/risk-scoring/agent-risk-workflow.md" target="_blank" rel="noopener noreferrer">Agent Risk Workflow</a></li> | ||
| </ul> | ||
| <h3><span aria-hidden="true">🔍</span> ${deepLabel}</h3> | ||
| <ul> | ||
| <li><a href="${analysisFileBase}/existing/deep-analysis.md" target="_blank" rel="noopener noreferrer">Deep Analysis</a></li> | ||
| <li><a href="${analysisFileBase}/existing/stakeholder-analysis.md" target="_blank" rel="noopener noreferrer">Stakeholder Analysis</a></li> | ||
| <li><a href="${analysisFileBase}/existing/coalition-analysis.md" target="_blank" rel="noopener noreferrer">Coalition Analysis</a></li> | ||
| <li><a href="${analysisFileBase}/existing/voting-patterns.md" target="_blank" rel="noopener noreferrer">Voting Patterns</a></li> | ||
| <li><a href="${analysisFileBase}/existing/cross-session-intelligence.md" target="_blank" rel="noopener noreferrer">Cross-Session Intelligence</a></li> | ||
| </ul> |
There was a problem hiding this comment.
These Analysis & Transparency links are constructed as analysis/{date}/{slug}. With suffix-based deduplication in the analysis stage (e.g. analysis/{date}/{slug}-2), articles generated on repeated runs will link to the wrong (earlier) analysis artifacts. Consider passing a run-specific analysis directory name into the template so it can link to the resolved analysis output path for that run.
| <li><a href="${analysisFileBase}/risk-scoring/risk-matrix.md" target="_blank" rel="noopener noreferrer">Risk Matrix</a></li> | ||
| <li><a href="${analysisFileBase}/risk-scoring/quantitative-swot.md" target="_blank" rel="noopener noreferrer">Quantitative SWOT</a></li> | ||
| <li><a href="${analysisFileBase}/risk-scoring/political-capital-risk.md" target="_blank" rel="noopener noreferrer">Political Capital Risk</a></li> | ||
| <li><a href="${analysisFileBase}/risk-scoring/legislative-velocity-risk.md" target="_blank" rel="noopener noreferrer">Legislative Velocity Risk</a></li> | ||
| <li><a href="${analysisFileBase}/risk-scoring/agent-risk-workflow.md" target="_blank" rel="noopener noreferrer">Agent Risk Workflow</a></li> | ||
| </ul> | ||
| <h3><span aria-hidden="true">🔍</span> ${deepLabel}</h3> | ||
| <ul> | ||
| <li><a href="${analysisFileBase}/existing/deep-analysis.md" target="_blank" rel="noopener noreferrer">Deep Analysis</a></li> | ||
| <li><a href="${analysisFileBase}/existing/stakeholder-analysis.md" target="_blank" rel="noopener noreferrer">Stakeholder Analysis</a></li> | ||
| <li><a href="${analysisFileBase}/existing/coalition-analysis.md" target="_blank" rel="noopener noreferrer">Coalition Analysis</a></li> | ||
| <li><a href="${analysisFileBase}/existing/voting-patterns.md" target="_blank" rel="noopener noreferrer">Voting Patterns</a></li> | ||
| <li><a href="${analysisFileBase}/existing/cross-session-intelligence.md" target="_blank" rel="noopener noreferrer">Cross-Session Intelligence</a></li> |
There was a problem hiding this comment.
The new per-file analysis links use English-only link text (e.g. “Significance Assessment”, “Actor Mapping”) regardless of the article language (lang). Since surrounding headings are localized and the HTML document sets lang, consider localizing these link titles (or reusing existing localized labels) to avoid mixed-language navigation for screen readers and non-English readers.
| <li><a href="${analysisFileBase}/risk-scoring/risk-matrix.md" target="_blank" rel="noopener noreferrer">Risk Matrix</a></li> | |
| <li><a href="${analysisFileBase}/risk-scoring/quantitative-swot.md" target="_blank" rel="noopener noreferrer">Quantitative SWOT</a></li> | |
| <li><a href="${analysisFileBase}/risk-scoring/political-capital-risk.md" target="_blank" rel="noopener noreferrer">Political Capital Risk</a></li> | |
| <li><a href="${analysisFileBase}/risk-scoring/legislative-velocity-risk.md" target="_blank" rel="noopener noreferrer">Legislative Velocity Risk</a></li> | |
| <li><a href="${analysisFileBase}/risk-scoring/agent-risk-workflow.md" target="_blank" rel="noopener noreferrer">Agent Risk Workflow</a></li> | |
| </ul> | |
| <h3><span aria-hidden="true">🔍</span> ${deepLabel}</h3> | |
| <ul> | |
| <li><a href="${analysisFileBase}/existing/deep-analysis.md" target="_blank" rel="noopener noreferrer">Deep Analysis</a></li> | |
| <li><a href="${analysisFileBase}/existing/stakeholder-analysis.md" target="_blank" rel="noopener noreferrer">Stakeholder Analysis</a></li> | |
| <li><a href="${analysisFileBase}/existing/coalition-analysis.md" target="_blank" rel="noopener noreferrer">Coalition Analysis</a></li> | |
| <li><a href="${analysisFileBase}/existing/voting-patterns.md" target="_blank" rel="noopener noreferrer">Voting Patterns</a></li> | |
| <li><a href="${analysisFileBase}/existing/cross-session-intelligence.md" target="_blank" rel="noopener noreferrer">Cross-Session Intelligence</a></li> | |
| <li><a href="${analysisFileBase}/risk-scoring/risk-matrix.md" target="_blank" rel="noopener noreferrer">${getLocalizedString(lang, { en: 'Risk Matrix', sv: 'Riskmatris', da: 'Risikomatrix', no: 'Risikomatrise', fi: 'Riskimatriisi', de: 'Risikomatrix', fr: 'Matrice des risques', es: 'Matriz de riesgos', nl: 'Risicomatrix', ar: 'مصفوفة المخاطر', he: 'מטריצת סיכונים', ja: 'リスクマトリクス', ko: '위험 매트릭스', zh: '风险矩阵' })}</a></li> | |
| <li><a href="${analysisFileBase}/risk-scoring/quantitative-swot.md" target="_blank" rel="noopener noreferrer">${getLocalizedString(lang, { en: 'Quantitative SWOT', sv: 'Kvantitativ SWOT', da: 'Kvantitativ SWOT', no: 'Kvantitativ SWOT', fi: 'Kvantitatiivinen SWOT', de: 'Quantitative SWOT', fr: 'SWOT quantitative', es: 'SWOT cuantitativo', nl: 'Kwantitatieve SWOT', ar: 'تحليل SWOT كمي', he: 'ניתוח SWOT כמותי', ja: '定量的SWOT', ko: '정량적 SWOT', zh: '定量 SWOT 分析' })}</a></li> | |
| <li><a href="${analysisFileBase}/risk-scoring/political-capital-risk.md" target="_blank" rel="noopener noreferrer">${getLocalizedString(lang, { en: 'Political Capital Risk', sv: 'Risk för politiskt kapital', da: 'Risiko for politisk kapital', no: 'Risiko for politisk kapital', fi: 'Poliittisen pääoman riski', de: 'Risiko politischen Kapitals', fr: 'Risque de capital politique', es: 'Riesgo de capital político', nl: 'Risico op politiek kapitaal', ar: 'مخاطر رأس المال السياسي', he: 'סיכון הון פוליטי', ja: '政治資本リスク', ko: '정치적 자본 위험', zh: '政治资本风险' })}</a></li> | |
| <li><a href="${analysisFileBase}/risk-scoring/legislative-velocity-risk.md" target="_blank" rel="noopener noreferrer">${getLocalizedString(lang, { en: 'Legislative Velocity Risk', sv: 'Risk för lagstiftningstakt', da: 'Risiko for lovgivningshastighed', no: 'Risiko for lovgivningstempo', fi: 'Lainsäädäntönopeuden riski', de: 'Risiko der Gesetzgebungsgeschwindigkeit', fr: 'Risque de vitesse législative', es: 'Riesgo de velocidad legislativa', nl: 'Risico op wetgevingssnelheid', ar: 'مخاطر وتيرة التشريع', he: 'סיכון קצב חקיקה', ja: '立法速度リスク', ko: '입법 속도 위험', zh: '立法速度风险' })}</a></li> | |
| <li><a href="${analysisFileBase}/risk-scoring/agent-risk-workflow.md" target="_blank" rel="noopener noreferrer">${getLocalizedString(lang, { en: 'Agent Risk Workflow', sv: 'Agentarbetsflöde för risk', da: 'Agentarbejdsgang for risiko', no: 'Agentarbeidsflyt for risiko', fi: 'Agentin riskityönkulku', de: 'Agenten-Risiko-Workflow', fr: 'Flux de travail de risque de l’agent', es: 'Flujo de trabajo de riesgo del agente', nl: 'Agent-risicowerkstroom', ar: 'سير عمل مخاطر الوكيل', he: 'תהליך עבודה לסיכוני סוכן', ja: 'エージェントリスクワークフロー', ko: '에이전트 위험 워크플로', zh: '代理风险工作流' })}</a></li> | |
| </ul> | |
| <h3><span aria-hidden="true">🔍</span> ${deepLabel}</h3> | |
| <ul> | |
| <li><a href="${analysisFileBase}/existing/deep-analysis.md" target="_blank" rel="noopener noreferrer">${getLocalizedString(lang, { en: 'Deep Analysis', sv: 'Fördjupad analys', da: 'Dybdeanalyse', no: 'Dybdanalyse', fi: 'Syväanalyysi', de: 'Tiefenanalyse', fr: 'Analyse approfondie', es: 'Análisis en profundidad', nl: 'Diepgaande analyse', ar: 'تحليل معمق', he: 'ניתוח מעמיק', ja: '詳細分析', ko: '심층 분석', zh: '深度分析' })}</a></li> | |
| <li><a href="${analysisFileBase}/existing/stakeholder-analysis.md" target="_blank" rel="noopener noreferrer">${getLocalizedString(lang, { en: 'Stakeholder Analysis', sv: 'Intressentanalys', da: 'Interessentanalyse', no: 'Interessentanalyse', fi: 'Sidosryhmäanalyysi', de: 'Stakeholder-Analyse', fr: 'Analyse des parties prenantes', es: 'Análisis de partes interesadas', nl: 'Stakeholderanalyse', ar: 'تحليل أصحاب المصلحة', he: 'ניתוח בעלי עניין', ja: 'ステークホルダー分析', ko: '이해관계자 분석', zh: '利益相关者分析' })}</a></li> | |
| <li><a href="${analysisFileBase}/existing/coalition-analysis.md" target="_blank" rel="noopener noreferrer">${getLocalizedString(lang, { en: 'Coalition Analysis', sv: 'Koalitionsanalys', da: 'Koalitionsanalyse', no: 'Koalisjonsanalyse', fi: 'Koalitioanalyysi', de: 'Koalitionsanalyse', fr: 'Analyse des coalitions', es: 'Análisis de coaliciones', nl: 'Coalitieanalyse', ar: 'تحليل الائتلافات', he: 'ניתוח קואליציות', ja: '連立分析', ko: '연합 분석', zh: '联盟分析' })}</a></li> | |
| <li><a href="${analysisFileBase}/existing/voting-patterns.md" target="_blank" rel="noopener noreferrer">${getLocalizedString(lang, { en: 'Voting Patterns', sv: 'Röstmönster', da: 'Stemmemønstre', no: 'Stemmemønstre', fi: 'Äänestysmallit', de: 'Abstimmungsmuster', fr: 'Schémas de vote', es: 'Patrones de votación', nl: 'Stempatronen', ar: 'أنماط التصويت', he: 'דפוסי הצבעה', ja: '投票パターン', ko: '투표 패턴', zh: '投票模式' })}</a></li> | |
| <li><a href="${analysisFileBase}/existing/cross-session-intelligence.md" target="_blank" rel="noopener noreferrer">${getLocalizedString(lang, { en: 'Cross-Session Intelligence', sv: 'Tvärsessionsintelligens', da: 'Tværsessionsintelligens', no: 'Tverrsesjonsintelligens', fi: 'Istuntojen välinen tiedusteluanalyysi', de: 'Sitzungsübergreifende Analyse', fr: 'Analyse intersessions', es: 'Inteligencia entre sesiones', nl: 'Sessie-overstijgende analyse', ar: 'تحليل عبر الجلسات', he: 'מודיעין בין-מושבי', ja: '会期間インテリジェンス', ko: '세션 간 인텔리전스', zh: '跨会期情报分析' })}</a></li> |
|
@copilot apply changes based on the comments in this thread and investigate failing checks |
|
@copilot apply changes based on the comments in this thread |
|
@copilot apply changes based on the comments in this thread |
|
@pethers Unfortunately I hit an unexpected error while processing your comment. I've automatically reported this to GitHub. You can ask me to try again later by mentioning me in a new comment. If you want to contact GitHub about this error, please mention the following identifier so they can better serve you: Sorry for the inconvenience! |
…ming, JSDoc, guide example Agent-Logs-Url: https://github.com/Hack23/euparliamentmonitor/sessions/9d3f226a-0aa3-4237-a29a-62ca2651ab04 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
All 5 review comments addressed in commit
|
| // If the analysis directory was deduplicated (e.g. "breaking-2"), extract | ||
| // just the numeric suffix and apply it to the current strategy's type. | ||
| // This keeps per-strategy slugs distinct when multiple article types are | ||
| // generated in a single run (e.g. "breaking-2" and "week-ahead-2"). | ||
| const suffixMatch = analysisDir?.match(/-(\d+)$/); | ||
| const dedupSuffix = suffixMatch ? `-${suffixMatch[1]}` : ''; |
There was a problem hiding this comment.
dedupSuffix extraction only supports numeric -(\d+) suffixes. If resolveUniqueAnalysisDir() falls back to the UUID suffix (or any future non-numeric suffix), article slugs won’t include the suffix, so repeated runs can still overwrite news/{date}-{type}-{lang}.html and the article’s canonical/og:url won’t line up with the deduplicated analysis directory.
Consider propagating the full suffix from the resolved analysis directory (numeric or UUID) and appending that to each strategy.type, instead of restricting to digits only.
| // If the analysis directory was deduplicated (e.g. "breaking-2"), extract | |
| // just the numeric suffix and apply it to the current strategy's type. | |
| // This keeps per-strategy slugs distinct when multiple article types are | |
| // generated in a single run (e.g. "breaking-2" and "week-ahead-2"). | |
| const suffixMatch = analysisDir?.match(/-(\d+)$/); | |
| const dedupSuffix = suffixMatch ? `-${suffixMatch[1]}` : ''; | |
| // If the analysis directory was deduplicated (e.g. "breaking-2" or a | |
| // UUID-based fallback such as "breaking-550e8400-e29b-41d4-a716-446655440000"), | |
| // propagate the full suffix for the current strategy type into the slug. | |
| // This keeps per-strategy slugs distinct and aligned with the resolved | |
| // analysis directory regardless of suffix format. | |
| const dedupPrefix = `${strategy.type}-`; | |
| const dedupSuffix = | |
| analysisDir !== undefined && analysisDir.startsWith(dedupPrefix) | |
| ? analysisDir.slice(strategy.type.length) | |
| : ''; |
|
@copilot apply changes based on the comments in this thread and analyse Code scanning results / CodeQL |
…Strategy Agent-Logs-Url: https://github.com/Hack23/euparliamentmonitor/sessions/eaee18d3-7f31-4a9c-affb-dc9e2578ebfd Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
…pport UUID dedup suffixes Agent-Logs-Url: https://github.com/Hack23/euparliamentmonitor/sessions/eaee18d3-7f31-4a9c-affb-dc9e2578ebfd Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
All changes applied in commits
Addressed in commits |
| // resolveUniqueAnalysisDir appends a numeric suffix (-2, -3, …) when | ||
| // a prior completed run (with manifest.json) already occupies the path, | ||
| // preventing repeated workflow runs from overwriting previous analysis. | ||
| const preferredDir = articleTypeSlug | ||
| ? path.resolve(outputDir, date, articleTypeSlug) | ||
| : path.resolve(outputDir, date); | ||
| const dateOutputDir = resolveUniqueAnalysisDir(preferredDir); | ||
|
|
There was a problem hiding this comment.
runAnalysisStage() calls resolveUniqueAnalysisDir(preferredDir) before validating hasSubstantiveData(fetchedData). Since resolveUniqueAnalysisDir may create a suffixed directory immediately, a run that later aborts (e.g. requireData=true with empty/invalid data, or another preflight error) can leave behind an empty claimed directory (e.g. .../breaking-2) that forces subsequent runs to skip to -3, etc. Consider moving the substantive-data validation earlier (before claiming/creating any directories), or cleaning up the claimed directory on early failure.
| function deriveTypeSlug(strategyType: string, analysisDir?: string): string { | ||
| const prefix = `${strategyType}-`; | ||
| if (analysisDir !== undefined && analysisDir.startsWith(prefix)) { | ||
| return analysisDir; | ||
| } | ||
| return strategyType; |
There was a problem hiding this comment.
deriveTypeSlug() returns the full analysisDir when it starts with ${strategyType}-, which breaks when analysisDir is a multi-type slug produced by deriveArticleTypeSlug (e.g. breaking-week-ahead-2): the “breaking” article would incorrectly get typeSlug breaking-week-ahead-2. It also does not implement the PR’s stated behavior of extracting the dedup suffix (e.g. -2) and applying it per strategy type. Consider parsing the suffix from analysisDir (numeric/UUID) and appending it to each strategyType, instead of substituting analysisDir wholesale.
| @@ -534,7 +539,14 @@ async function main(): Promise<void> { | |||
| } | |||
|
|
|||
| results.push( | |||
| await generateArticleForStrategy(strategy, client, languages, outputOptions, stats) | |||
| await generateArticleForStrategy( | |||
| strategy, | |||
| client, | |||
| languages, | |||
| outputOptions, | |||
| stats, | |||
| analysisDir | |||
| ) | |||
There was a problem hiding this comment.
news-enhanced.ts passes a single analysisDir (basename of analysisCtx.outputDir) to generateArticleForStrategy() for every requested article type. When multiple articleTypes are generated in one invocation, the analysis output directory slug is derived from all types (deriveArticleTypeSlug), so analysisDir will not match each individual strategy’s analysis folder. This will make Analysis & Transparency links (and potentially the deduped type slug logic) point to the wrong directory for some articles. A safer approach is to compute a per-strategy analysisDir (or just a suffix) and pass that per strategy instead of reusing the run-level basename for all strategies.
| export function resolveUniqueAnalysisDir(baseDir: string): string { | ||
| // If the directory doesn't exist yet or has no manifest from a prior | ||
| // completed run, use it as-is. A directory without manifest.json is | ||
| // considered available (not yet finished by any run). | ||
| if (!fs.existsSync(path.join(baseDir, 'manifest.json'))) { | ||
| return baseDir; |
There was a problem hiding this comment.
PR description says “Any existing directory … is treated as occupied” for slug-level dedup, but resolveUniqueAnalysisDir() currently treats a directory as occupied only when manifest.json exists (and even comments that a directory without manifest is considered available). If the intended contract is “any existing dir is occupied”, the implementation and JSDoc/comment here should be updated to match (or the PR description adjusted).
| /** | ||
| * Extract the deduplication suffix from a resolved analysis directory name | ||
| * and apply it to the given strategy type. | ||
| * | ||
| * If `analysisDir` is `"breaking-2"` and `strategyType` is `"breaking"`, | ||
| * returns `"breaking-2"`. For UUID-based fallbacks like `"breaking-a1b2c3d4"`, | ||
| * returns `"breaking-a1b2c3d4"`. When no suffix applies, returns the | ||
| * `strategyType` unchanged. | ||
| * | ||
| * @param strategyType - Base article type (e.g. `"breaking"`) | ||
| * @param analysisDir - Resolved analysis directory basename, if any | ||
| * @returns The type slug with any dedup suffix appended | ||
| */ |
There was a problem hiding this comment.
The new dedup-related behavior (deriveTypeSlug + propagation of analysisDir into generateArticleHTML for canonical URLs / transparency links) is not covered by unit tests. There are existing generate-stage pipeline tests, but none assert that a suffixed analysis run (e.g. breaking-2) results in (a) the expected article slug/filename pattern and (b) transparency links pointing at the suffixed analysis directory. Adding focused Vitest cases for numeric and UUID suffixes (and for multi-article-type slugs) would help prevent regressions.
|
@copilot apply changes based on the comments in this thread |
…ing, move data validation before dir claiming, add dedup unit tests - deriveTypeSlug now takes a dedupSuffix string (e.g. "-2") instead of full analysisDir, fixing multi-type slug corruption - news-enhanced.ts extracts dedupSuffix by comparing resolved dir basename with base slug - resolveUniqueAnalysisDir uses manifest.json check for base dir (supports skipCompleted) + atomic mkdirSync for suffixed candidates - analysis-stage.ts moves hasSubstantiveData check before resolveUniqueAnalysisDir to prevent orphan dirs on abort - Added 11 focused Vitest tests for deriveTypeSlug, generateArticleForStrategy with dedup, and resolveUniqueAnalysisDir Agent-Logs-Url: https://github.com/Hack23/euparliamentmonitor/sessions/07cc3829-1346-4465-8560-d2fabe1befc6 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Agent-Logs-Url: https://github.com/Hack23/euparliamentmonitor/sessions/07cc3829-1346-4465-8560-d2fabe1befc6 Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
All 5 review comments addressed in commits
CodeQL: All 14 alerts are pre-existing |
|
@copilot analyse and fix Test and Report / Validate Code (pull_request) |
Agent-Logs-Url: https://github.com/Hack23/euparliamentmonitor/sessions/bbc94201-af13-4c47-846a-bd566a13357d Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
All 5 review comments addressed in commits
CI failure was a Prettier formatting issue in |
Repeated same-day workflow runs (e.g. breaking news every 6h) silently overwrote prior analysis directories and news articles, destroying committed intelligence. Additionally, the previous PR's improvements to AI analysis guides, workflow prompts, and article template analysis links are included.
Slug-level suffix deduplication
Two new utilities in
src/utils/file-utils.ts:resolveUniqueAnalysisDir(baseDir)— uses aclaimAnalysisDir()helper with atomicfs.mkdirSync({ recursive: false })and EEXIST retry to race-safely claim suffixed directories. The base directory usesmanifest.jsonpresence to determine occupancy (supporting incremental/skipCompleted runs), while suffixed candidates (-2,-3, …) use atomicmkdirSyncclaiming. UUID-based fallback (randomUUID().slice(0,8)) for overflow.resolveUniqueFilePath(filepath)— appends-2,-3, … before the extension when the file exists, with UUID-based fallback.Deduplication operates at the slug level, not the file level. The dedup suffix is computed by comparing the resolved analysis directory basename with the original base slug, then passed separately as
dedupSuffix(e.g."-2") to each strategy. This ensures multiple article types in one run retain distinct slugs (e.g.breaking-2,week-ahead-2) without multi-type slug corruption. The suffix is validated against a dedup pattern (/^-[\da-f]+$/i) to support both numeric and UUID formats. The suffixed slug is baked into the article before HTML generation, so filenames, canonical URLs,og:url, and language-switcher links all stay consistent:The existing
ARTICLE_FILENAME_PATTERNregex matches deduplicated filenames without modification, since the-2suffix falls within the slug capture group(.+).analysis-stage.ts: validateshasSubstantiveData(fetchedData)before callingresolveUniqueAnalysisDirto prevent orphan claimed directories on early abort; resolves unique dir via atomicclaimAnalysisDirbeforeensureDirectoryExistsgenerate-stage.ts:deriveTypeSlug(strategyType, dedupSuffix)helper appends the dedup suffix to each strategy type independently, keeping per-strategy slugs distinct; passes the suffixed slug as bothslugandanalysisDirintogenerateArticleHTML()so canonical/og:url/language-switcher links match the written filenameoutput-stage.ts:writeArticleFilereturns the actual written filename (string | null) instead ofboolean, with no file-level dedup (slug-level dedup upstream ensures uniqueness); JSDoc aligned to reflect actual behaviornews-enhanced.ts: computesdedupSuffixby comparing resolved dir basename with original slug, passes bothdedupSuffixandanalysisDirper-strategy throughgenerateArticleForStrategy()Analysis directory provenance in article links
ArticleOptions.analysisDir— new optional field that overrides the slug when constructing Analysis & Transparency links, so articles generated from a suffixed analysis run (e.g.breaking-2) link to the correct analysis directory instead of the originalrenderAnalysisTransparencySectionaccepts and uses the resolved analysis directory name through the pipelinenews-enhanced.tscapturesAnalysisContextfromrunAnalysisWithGuard(), extracts the resolved directory basename viapath.basename(analysisCtx.outputDir), and passes it asanalysisDirthroughgenerateArticleForStrategy()→generateSingleLanguageArticle()→generateArticleHTML()→renderAnalysisTransparencySection(), ensuring articles always link to the actual analysis output directorymanifest.jsonfor run provenance per AI Analysis Guide Rule 10Localized per-file analysis link labels
All 18 individual analysis file links (classification, threat assessment, risk scoring, deep analysis sections) now use fully localized labels across all 14 supported languages via
getLocalizedString()and dedicatedLanguageMapconstants inlanguage-ui.ts, fixing the English-only regression for non-English articles and screen readers.Unit test coverage for dedup behavior
Added 11 focused Vitest cases covering:
deriveTypeSlugwith numeric suffixes, UUID suffixes, empty suffix, and per-strategy distinctnessgenerateArticleForStrategywith dedup suffix propagation to filenames/slugsresolveUniqueAnalysisDirbase dir behavior, suffixed candidate claiming, manifest.json occupancy check, and no-manifest reuseAI analysis quality improvements (from prior session)
breaking-2-en.html, notbreaking-en-2.html)..mdfiles plusmanifest.jsoninstead of 5 directory-level linksexisting/methods (deep-analysis, stakeholder, coalition, voting-patterns, cross-session) now emit structured AI prompt templates instead of bare data inventory tables📰 AI-DRIVEN HEADLINE AND DESCRIPTION GENERATIONsections with rejected/required title examples