fix(ci): coordinate qwen triage and review automation#4570
Conversation
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
0c7e61a to
7385b70
Compare
|
@yiliang114 same goal, discuss and merge one of them. #4577 |
|
Excited to get AI doing this for us |
99f4e62 to
507695f
Compare
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Pull request overview
Adds a new maintainer-facing /triage skill under .qwen/skills/triage/ to standardize PR intake and issue triage (including gating for safe GitHub side effects), and updates the issue follow-up bot workflow to skip items already handled by maintainer triage markers.
Changes:
- Introduces the
/triageskill spec (SKILL.md) including a staged-report workflow, side-effect gates, PR intake rules, and issue triage rules. - Adds supporting triage reference docs (workflow overview + tone guide + rulebooks).
- Extends the follow-up bot’s marker skip list to include
qwen-issue-bot:welcome-prandqwen-maintain:*markers to prevent duplicate/conflicting comments.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
.qwen/skills/triage/SKILL.md |
Defines /triage routing, gates, staged report format, and safe execution guidance for PRs/issues. |
.qwen/skills/triage/references/workflow-overview.md |
Describes the skill architecture and marker coordination with the follow-up bot. |
.qwen/skills/triage/references/tone-guide.md |
Sets tone/structure expectations for public-facing triage comments. |
.qwen/skills/triage/references/pr-intake-rules.md |
Rulebook for PR intake readiness evaluation and suggested label taxonomy. |
.qwen/skills/triage/references/issue-triage-rules.md |
Rulebook for issue classification, completeness checks, diagnosis routing, and markers. |
.github/workflows/qwen-issue-followup-bot.yml |
Adds triage markers to the bot’s “already handled” skip list. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
wenshao
left a comment
There was a problem hiding this comment.
Note (forward-looking, non-blocking — no single diff line to anchor to): SKILL.md says the skill is "designed to run unattended in CI" with "no human confirmation needed at runtime," yet all its safety controls are natural-language instructions. The followup bot it coordinates with enforces its rails mechanically (a safe_gh shim pinning --repo, a coreTools allowlist, secret-redaction), and the skill can additionally call gh api ... --method PATCH, which the bot's shim forbids. Separately, Step 0 validates the target is numeric only on the bare-number branch (lines 61-72), not the URL branch (58-59). None of this is wired to CI in this PR, so run interactively by a maintainer the risk is low — but if it is ever automated, consider the same enforced layer plus numeric validation/normalization on the URL branch.
— claude-opus-4-8 via Claude Code /qreview
…nd bot coordination Builds on #4577's triage skill with: - Tiered gate model (comment gate + label gate) to prevent noise on already-reviewed PRs and avoid duplicate labels - PR/Issue auto-detection for numeric inputs - Dual marker coordination with followup bot (qwen-issue-bot + qwen-maintain) - Issue triage rules: P0-P3 priority, completeness check, version staleness, auto-fix/welcome-PR eligibility, related vs duplicate search strategy - PR intake rules: author validation table by PR type, scope thresholds (800/1500), deep review handoff conditions, label taxonomy mapping - Comment anti-patterns and public comment distillation - Bot workflow skip list updated with 3 new markers
Triggers: - Issue opened → auto triage - PR opened → auto triage - Comment `@qwen /triage` → re-triage (maintainers only) - Manual workflow_dispatch Uses qwen-code-action to invoke the triage skill, following the same pattern as qwen-code-pr-review.yml.
1fcfd23 to
f972389
Compare
This comment has been minimized.
This comment has been minimized.
|
Current state after the latest update:
I updated the PR body to match this scope. Older comments that described the temporary pre-adjustment event split should be treated as outdated. |
|
PR scope reduced and title/body updated to match the current 2-file diff (workflow hardening only). All review threads resolved:
|
Evidence for the current scopeAdding the concrete runs/reviews that motivated the current 3-file diff:
Additional cleanup/hardening included in the same small diff:
The current PR intentionally does not merge issue follow-up, scheduled issue triage, release labeling, or label cleanup. Those are separate workflow-policy questions. |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Thanks for the PR @yiliang114! The direction here is solid — coordinating triage and review to avoid conflicting signals is a real operational need, and the concurrency/timeout fixes address genuine production issues.
However, the PR body is missing several required sections from the PR template:
## Reviewer Test Plan— the PR has## Test planbut is missing the required sub-sections:### How to verify,### Evidence (Before & After), and### Tested ontable. Even for CI-only changes, these should be present (mark N/A where applicable).## Linked Issues— issues are referenced inline ("Related to #4577, #4768, #4787") but the formal## Linked Issuessection is absent.<details>中文说明</details>— the bilingual summary block is missing entirely.## Risk & Scope— heading casing differs and the required bullet structure (Main risk or tradeoff,Not validated / out of scope,Breaking changes / migration notes) isn't followed.
Please update the PR body to match the template. The actual content is good — it's a structural/formatting pass needed for consistency.
中文说明
感谢 @yiliang114 的贡献!方向是对的——协调 triage 和 review 避免冲突信号是真实的运维需求,并发/超时修复也解决了生产中的实际问题。
但 PR 正文缺少模板要求的几个章节:
## Reviewer Test Plan— PR 用的是## Test plan,缺少必须的子章节:### How to verify、### Evidence (Before & After)和### Tested on表格。即使是纯 CI 改动,也应保留这些章节(不适用的标 N/A)。## Linked Issues— issue 在正文中引用了("Related to #4577, #4768, #4787"),但缺少正式的## Linked Issues章节。<details>中文说明</details>— 完全缺少中文摘要块。## Risk & Scope— 标题大小写不一致,且未使用要求的 bullet 结构(Main risk or tradeoff、Not validated / out of scope、Breaking changes / migration notes)。
请按模板更新 PR 正文。内容本身没问题,只是需要结构/格式上的调整。
— Qwen Code · qwen3.7-max
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
No review findings. Downgraded from Approve to Comment: CI still running.
The workflow coordination changes are well-structured: conditional cancel-in-progress expressions are consistent with their respective if conditions, the fork guard and draft check properly scope pull_request_target, removing automatic /review on PR open eliminates the triage-vs-review race documented in the PR body, and the Stage 1a terminal gate clarification reduces noisy duplicate outputs. Previous review feedback (fork guard, actions: write removal, conditional cancellation, ready_for_review trigger, maxSessionTurns rationale) has been addressed with clear evidence.
— qwen3.7-max via Qwen Code /review
|
@qwen-code /triage |
|
Thanks for the PR @yiliang114! Template looks good ✓ — all required sections present and filled in properly. On direction: this is solid CI hygiene. The problems are real and well-evidenced — turn-cap exit code 53 killing triage mid-run, 10-minute timeout too tight for full Stage 2/3, bot comments racing with concurrency. These are the kind of production-warts fixes that only surface after the workflow has been running for a while. Clearly aligned with the project's triage automation investment. On approach: the scope is tight — 2 files, +29/-9, each change traceable to a specific observed failure. The One thing worth confirming: with
Moving on to code review. 🔍 中文说明感谢贡献 @yiliang114! 模板完整 ✓ — 所有章节齐全且填写规范。 方向:这是扎实的 CI 维护工作。问题都是真实的且有生产证据支撑——轮数上限导致 exit code 53 中断 triage、10 分钟超时对完整 Stage 2/3 不够用、bot 评论与并发竞争。这类生产环境的磨损修复只有在工作流运行一段时间后才会暴露,与项目对 triage 自动化的投入完全一致。 方案:范围紧凑——2 个文件、+29/-9,每处改动都可追溯到具体的观测故障。 值得确认一点:
进入代码审查 🔍 — Qwen Code · qwen3.7-max |
Code ReviewTwo files changed, both straightforward. No correctness bugs found.
ValidationTmux TestingN/A — this is a CI-only workflow change with no user-visible CLI behavior to demonstrate. The changes affect GitHub Actions trigger conditions, concurrency groups, and timeout settings. There is no terminal interaction to capture. 中文说明代码审查两个文件改动,都很直接。未发现正确性 bug。
验证Tmux 测试不适用——这是纯 CI 工作流变更,无可演示的用户可见 CLI 行为。改动影响 GitHub Actions 触发条件、并发组和超时设置,没有可捕获的终端交互。 — Qwen Code · qwen3.7-max |
ReflectionThis PR solves real, well-documented production problems with the triage workflow. The diff is small (+29/-9), each change traces back to a specific observed failure, and the code review found no bugs. The concurrency/cancellation logic is the trickiest part and it's handled correctly — the The existing My independent assessment: this is the right fix at the right scope. Nothing to cut, nothing to add. Approving. ✅ 中文说明总结此 PR 解决了 triage 工作流中真实的、有充分记录的生产问题。diff 很小(+29/-9),每处改动都可追溯到具体的观测故障,代码审查未发现 bug。 并发/取消逻辑是最复杂的部分,处理正确—— 此 PR 上现有的 我的独立评估:这是正确范围内的正确修复。无需删减,无需增加。 批准 ✅ — Qwen Code · qwen3.7-max |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
Reflection
This is a well-motivated workflow hardening PR. The problems it solves are real and well-documented with production CI evidence. The diff is small, focused, and doesn't try to do more than it needs to.
The implementation quality is solid — the concurrency/cancel-in-progress duplication is intentional and correctly explained, the draft guard and same-repo restriction are appropriate security hardening, and the ready_for_review trigger fills a genuine gap.
Two items need resolution before merge:
-
maxSessionTurnsremoval — wenshao's concern is valid. The 30-min wall clock is a softer bound than a turn limit. Consider raising the cap (50–75) rather than removing it entirely, or explicitly document that the workflow timeout is the chosen bound and accept the cost tradeoff. -
PR body mismatch — the body describes changes to
qwen-code-pr-review.ymlthat aren't in the diff. Minor, but worth updating so the description matches the actual scope.
Neither is a design problem — they're loose ends. Once resolved, this is ready.
中文说明
总结
这是一个动机良好的工作流加固 PR。它解决的问题是真实的,并用生产 CI 证据充分记录。diff 小、专注,没有试图做超出需要的事情。
实现质量扎实——并发/cancel-in-progress 的重复是有意的且正确解释,draft guard 和同仓库限制是适当的安全加固,ready_for_review 触发器填补了真正的空白。
合并前需要解决两个项目:
-
maxSessionTurns移除 — wenshao 的担忧是合理的。30 分钟墙钟时间是比轮数限制更软的约束。考虑提高上限(50-75)而不是完全移除,或明确记录工作流超时是选定的约束并接受成本权衡。 -
PR body 不匹配 — body 描述了不在 diff 中的
qwen-code-pr-review.yml改动。小问题,但值得更新以使描述与实际范围匹配。
两者都不是设计问题——它们是未收尾的细节。一旦解决,就可以合并。
— Qwen Code · qwen3.7-max
|
Re: maxSessionTurns — intentionally not adding it back. The 30-min workflow timeout is the single execution bound. Turn limits caused premature exit (code 53) before Stage 2/3 could complete, which was the original problem this PR fixes. The cost tradeoff is acceptable given the timeout acts as the hard stop. Re: PR body — updated to reflect the reverted |
|
@qwen-code /triage |
qwen-code-ci-bot
left a comment
There was a problem hiding this comment.
LGTM, looks ready to ship. ✅
What this PR does
Hardens the Qwen triage workflow so it handles real production scenarios (long-running triage, draft PRs, fork PRs, concurrent runs) without premature exits or race conditions.
Changes to
.github/workflows/qwen-triage.yml:maxSessionTurns: 25, letting the outer workflow timeout be the single hard stop for longer PR triage runs.@qwen-code /triagereruns to cancel an in-progress run.pull_request_targetto same-repository non-draft PRs and remove the unusedactions: writepermission.Changes to
.qwen/skills/triage/references/pr-workflow.md:CHANGES_REQUESTEDreview and stop, without also posting a Stage 1 issue comment or continuing to Stage 2/3.Why it's needed
This PR is based on the production behavior seen after the initial triage workflow landed. Several separate issues showed up in real runs: PR triage could hit exit code 53 from the 25-turn session cap before Stage 2/3 completed, the 10-minute workflow timeout was too tight for full triage with test evidence, and
issue_commentevents produced by bot comments could race with concurrency and cancel an issue-comment-triggered triage run before it finished.Template-check failures also needed a single public terminal output. If the PR body is missing required sections, triage should request changes once and stop rather than also emitting a Stage 1 comment that can read like a separate verdict.
Reviewer Test Plan
How to verify
actionlintpasses forqwen-triage.ymlwith the repository's existing self-hosted runner label warning ignored forecs-qwen.git diff --checkis clean.@qwen-code /triage, non-command comments, and bot-generated comments/reviews.Evidence (Before & After)
N/A — CI-only workflow changes, no user-visible UI.
Tested on
Risk & Scope
maxSessionTurnsmeans the 30-minute workflow timeout is the only execution bound. This is intentional — the turn limit caused premature exit before triage stages could complete.pull_request_targetPR triage — fork PRs can still be triaged manually. Review workflow changes are handled separately in Align automated PR review with bundled skill #4843.Linked Issues
Related to #4577, #4768, #4787.
中文说明
这个 PR 做了什么
加固 Qwen triage 工作流,使其能处理真实生产场景(长时间运行的 triage、draft PR、fork PR、并发运行)而不会过早退出或产生竞争条件。
.github/workflows/qwen-triage.yml的改动:maxSessionTurns: 25,让外层工作流超时作为长 PR triage 运行的唯一硬性停止条件。@qwen-code /triage重跑取消正在进行的运行。pull_request_target限制为同仓库非 draft PR,移除未使用的actions: write权限。.qwen/skills/triage/references/pr-workflow.md的改动:CHANGES_REQUESTEDreview 然后停止,不再额外发布 Stage 1 issue comment 或继续到 Stage 2/3。为什么需要
本 PR 基于初始 triage 工作流上线后的生产行为。多个独立问题在真实运行中出现:PR triage 可能在 Stage 2/3 完成前因 25-turn session cap 触发退出码 53,10 分钟工作流超时对带测试证据的完整 triage 过于紧张,bot 评论产生的
issue_comment事件可能与并发竞争并在 triage 完成前取消运行。模板检查失败也需要单一的公开终止输出。如果 PR body 缺少必要章节,triage 应一次性请求修改后停止,而非额外发出看起来像独立判定的 Stage 1 评论。
Reviewer Test Plan
如何验证
actionlint对qwen-triage.yml通过(忽略ecs-qwen的自托管 runner 标签警告)。git diff --check干净。@qwen-code /triage、非命令评论、bot 生成的评论/review。证据(前后对比)
N/A — 纯 CI 工作流变更,无用户可见 UI。
测试平台
风险与范围
maxSessionTurns意味着 30 分钟工作流超时是唯一的执行约束。这是有意的——轮数限制导致 triage 阶段无法完成就提前退出。pull_request_targetPR triage——fork PR 仍可由维护者手动 triage。Review workflow 的改动在 Align automated PR review with bundled skill #4843 中单独处理。关联 Issues
Related to #4577, #4768, #4787.