Skip to content

Step summary truncates agent output at 500 chars with no visible warning #19810

@danielmeppiel

Description

@danielmeppiel

Context

I'm building microsoft/apm-action — a GitHub Action that deploys agent primitives (instructions, prompts, skills, agents) into CI environments via steps: frontmatter in GH AW workflows.

After apm install, the Copilot agent prints a deployment summary table listing all installed primitives. In the step summary, only 1 out of 7 rows is visible — the rest is silently truncated.

Observed behavior

In run 22749832020, the Copilot agent's full output includes a 7-row table of deployed primitives. The step summary shows only row 1, ending with ... — which inside the code fence looks like part of the table content, not a truncation indicator.

Three things contribute to this:

1. $GITHUB_STEP_SUMMARY env var is a dangling reference inside the sandbox

The env var is inherited from the parent runner process, but the file it points to (e.g. /home/runner/_layout/_work/_temp/step_summary_<guid>) doesn't exist inside the awf sandbox's isolated filesystem. The agent tries cat >> "$GITHUB_STEP_SUMMARY", it fails silently, then the agent detects the failure and pivots to printing the output directly — consuming ~200 chars of error recovery text like:

"The GITHUB_STEP_SUMMARY environment variable points to a file that doesn't exist or isn't accessible in this environment. Let me output the summary directly instead..."

This recovery text has zero value for the user but consumes ~40% of the truncation budget.

2. Per-text-block truncation at 500 chars with no visible indicator

In log_parser_shared.cjs, generateCopilotCliStyleSummary() (~line 1270):

const maxTextLength = 500;
let displayText = text;
if (displayText.length > maxTextLength) {
  displayText = displayText.substring(0, maxTextLength) + "...";
}

The ... appended after truncation is indistinguishable from content when rendered inside a code fence. There's no ⚠️ banner or distinguishable marker. The existing SIZE_LIMIT_WARNING only fires when the overall 1MB step summary limit is reached — not for individual text-block truncation.

3. Failed bash command shows instead of

The cat >> "$GITHUB_STEP_SUMMARY" command fails (the file doesn't exist), but the step summary shows next to it. This is because scanForToolErrors() in parse_copilot_log.cjs only detects errors via [ERROR] log markers. Bash commands that fail without producing an [ERROR]-prefixed line default to is_error: false.

Impact

For actions that install or configure resources and report results via agent text output, the 500-char limit means structured output (tables, lists) with preamble text gets cut off before the useful content. The dangling $GITHUB_STEP_SUMMARY env var makes this worse by causing the agent to waste the budget on error recovery prose.

Workaround

We've worked around this in microsoft/apm-action#4 by outputting a compact one-liner first:

APM: 5 primitives deployed (2 prompts, 1 agent, 2 skills) + AGENTS.md

This ~70-char headline always survives the 500-char truncation. The detailed file listing follows as bonus info.

Environment

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions