feat: show dedicated message for cyber_policy_violation engine failures#24428
feat: show dedicated message for cyber_policy_violation engine failures#24428
Conversation
…gent failure issue Agent-Logs-Url: https://github.com/github/gh-aw/sessions/41025a85-2b7b-41f7-8541-42e63e526847 Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
…n template Agent-Logs-Url: https://github.com/github/gh-aw/sessions/74c64632-417e-47d2-825f-a8a50b25e795 Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Adds a dedicated, user-facing explanation when the Codex engine fails due to OpenAI’s cyber_policy_violation policy, replacing the previously raw/unhelpful token-only failure output.
Changes:
- Introduces a new markdown template explaining
cyber_policy_violationand suggested remediation steps. - Updates engine-failure context generation to detect
cyber_policy_violationand render the dedicated template (with fallback to the generic engine-failure block). - Adds test coverage for template-present, template-missing fallback, and multi-error scenarios.
Show a summary per file
| File | Description |
|---|---|
| actions/setup/md/cyber_policy_violation.md | New end-user guidance template for cyber_policy_violation engine blocks. |
| actions/setup/js/handle_agent_failure.cjs | Detects cyber_policy_violation and renders the dedicated template when available. |
| actions/setup/js/handle_agent_failure.test.cjs | Adds tests covering the new dedicated-message behavior and fallback. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comments suppressed due to low confidence (1)
actions/setup/js/handle_agent_failure.test.cjs:486
- afterEach unconditionally deletes RUNNER_TEMP, which can break other tests when the suite is run in the same worker process. Prefer restoring RUNNER_TEMP to its prior value (or only deleting it if it was originally undefined).
afterEach(() => {
delete process.env.GH_AW_AGENT_OUTPUT;
delete process.env.GH_AW_ENGINE_ID;
delete process.env.RUNNER_TEMP;
// Clean up temp dir
- Files reviewed: 3/3 changed files
- Comments generated: 3
| This restriction is usually temporary. To resolve this: | ||
|
|
||
| 1. Wait a few minutes and re-run the workflow | ||
| 2. Check your [OpenAI usage policies](https://platform.openai.com/docs/guides/safety-best-practices) for more information |
There was a problem hiding this comment.
The link text says "OpenAI usage policies" but the URL points to the "safety-best-practices" guide. Please either update the anchor text to match the target (e.g., "OpenAI safety best practices") or change the URL to the actual usage policies page so the guidance isn’t misleading.
| 2. Check your [OpenAI usage policies](https://platform.openai.com/docs/guides/safety-best-practices) for more information | |
| 2. Check your [OpenAI safety best practices](https://platform.openai.com/docs/guides/safety-best-practices) for more information |
| promptsDir = path.join(tmpDir, "gh-aw", "prompts"); | ||
| fs.mkdirSync(promptsDir, { recursive: true }); | ||
| process.env.GH_AW_AGENT_OUTPUT = path.join(tmpDir, "agent_output.json"); | ||
| process.env.RUNNER_TEMP = tmpDir; | ||
| ({ buildEngineFailureContext } = require("./handle_agent_failure.cjs")); |
There was a problem hiding this comment.
This test overwrites RUNNER_TEMP but doesn’t preserve the previous value. Please store the original RUNNER_TEMP (if any) in beforeEach and restore it in afterEach to avoid leaking environment changes into other test files that may rely on RUNNER_TEMP being set.
This issue also appears on line 482 of the same file.
| try { | ||
| return "\n" + renderTemplateFromFile(templatePath, {}); | ||
| } catch { | ||
| // Template not available — fall through to generic engine failure message | ||
| core.info(`cyber_policy_violation template not found at ${templatePath}, using generic message`); |
There was a problem hiding this comment.
The bare catch here will treat any read/render error as "template not found", which can hide real problems (e.g., permission issues, malformed template). Consider checking fs.existsSync(templatePath) before rendering, and/or catching the error as a variable and logging getErrorMessage(error) so unexpected failures are diagnosable while still falling back to the generic engine failure message.
| try { | |
| return "\n" + renderTemplateFromFile(templatePath, {}); | |
| } catch { | |
| // Template not available — fall through to generic engine failure message | |
| core.info(`cyber_policy_violation template not found at ${templatePath}, using generic message`); | |
| if (!fs.existsSync(templatePath)) { | |
| // Template not available — fall through to generic engine failure message | |
| core.info(`cyber_policy_violation template not found at ${templatePath}, using generic message`); | |
| } else { | |
| try { | |
| return "\n" + renderTemplateFromFile(templatePath, {}); | |
| } catch (error) { | |
| core.info(`Failed to render cyber_policy_violation template at ${templatePath}: ${getErrorMessage(error)}; using generic message`); | |
| } |
When the Codex engine is blocked by OpenAI's
cyber_policy_violationpolicy, the agent failure issue previously surfaced only the raw error token with no context or guidance.Changes
actions/setup/md/cyber_policy_violation.md— New template with a plain-language explanation of the error, why it occurs, and remediation steps (retry, review OpenAI account status, audit prompt content for policy-triggering patterns)actions/setup/js/handle_agent_failure.cjs—buildEngineFailureContext()now checks collected error messages forcyber_policy_violation; when present, renders the dedicated template instead of the generic "Engine Failure" block. Degrades gracefully to the existing behavior if the template file is absentactions/setup/js/handle_agent_failure.test.cjs— Tests for template-present, template-missing (fallback), and multi-error scenarios