Skip to content

feat: add exit code context for common CLI tools in terminal results#5144

Merged
teknium1 merged 1 commit into
mainfrom
hermes/hermes-8c57ca22
Apr 4, 2026
Merged

feat: add exit code context for common CLI tools in terminal results#5144
teknium1 merged 1 commit into
mainfrom
hermes/hermes-8c57ca22

Conversation

@teknium1

@teknium1 teknium1 commented Apr 4, 2026

Copy link
Copy Markdown
Contributor

Problem

When the agent runs grep, diff, test, or find via the terminal tool, a non-zero exit code gets returned raw. The model interprets exit_code: 1 as failure and burns a turn investigating — even though grep returning 1 just means "no matches found" and diff returning 1 means "files differ", both perfectly normal.

Our search_files tool already handles this correctly (it checks rg exit codes internally and returns clean total_count: 0), but there's no equivalent for raw terminal usage. The agent hits this when using diff (no dedicated tool), test in conditionals, or grep inside pipelines.

Solution

Add a lightweight _interpret_exit_code(command, exit_code) function that returns a human-readable note when a non-zero exit code has a known non-error meaning. The note is included as an exit_code_meaning field in the terminal JSON result — purely additive, the exit_code field itself is unchanged.

Before:

{"output": "", "exit_code": 1, "error": null}

After:

{"output": "", "exit_code": 1, "error": null, "exit_code_meaning": "No matches found (not an error)"}

Commands covered

Command Exit 1 meaning Higher codes
grep/egrep/fgrep/rg/ag/ack No matches found Real error (no note)
diff/colordiff Files differ Real error (no note)
find Some dirs inaccessible Real error (no note)
test / [ Condition is false Real error (no note)
curl 6=DNS, 7=connect, 22=HTTP error, 28=timeout
git Context-dependent (e.g. diff returns 1 when files differ)

Pipeline handling

The function extracts the last command in a pipeline or chain (since that determines the exit code):

  • ls -la | grep 'pattern' → interprets as grep
  • cd /tmp && diff a b → interprets as diff
  • /usr/bin/grep → strips path
  • LANG=C grep foo → strips env var prefix

Tests

31 new tests in test_terminal_exit_semantics.py covering all mappings, pipeline/chain parsing, path stripping, env var prefixes, unknown commands, and edge cases.

Existing terminal tests: 17 passed, 0 regressions.
Full tools suite: 2268 passed, 0 regressions (10 pre-existing failures in unrelated skill manager tests).

When commands like grep, diff, test, or find return non-zero exit codes
that aren't actual errors (grep 1 = no matches, diff 1 = files differ),
the model wastes turns investigating non-problems. This adds an
exit_code_meaning field to the terminal JSON result that explains
informational exit codes, so the agent can move on instead of debugging.

Covers grep/rg/ag/ack (no matches), diff (files differ), find (partial
access), test/[ (condition false), curl (timeouts, DNS, HTTP errors),
and git (context-dependent). Correctly extracts the last command from
pipelines and chains, strips full paths and env var assignments.

The exit_code field itself is unchanged — this is purely additive context.
@teknium1 teknium1 force-pushed the hermes/hermes-8c57ca22 branch from 9a9852a to 5619a83 Compare April 4, 2026 23:57
@teknium1 teknium1 merged commit aa475ae into main Apr 4, 2026
naoironman-hue pushed a commit to naoironman-hue/hermes-agent that referenced this pull request Apr 5, 2026
…ousResearch#5144)

When commands like grep, diff, test, or find return non-zero exit codes
that aren't actual errors (grep 1 = no matches, diff 1 = files differ),
the model wastes turns investigating non-problems. This adds an
exit_code_meaning field to the terminal JSON result that explains
informational exit codes, so the agent can move on instead of debugging.

Covers grep/rg/ag/ack (no matches), diff (files differ), find (partial
access), test/[ (condition false), curl (timeouts, DNS, HTTP errors),
and git (context-dependent). Correctly extracts the last command from
pipelines and chains, strips full paths and env var assignments.

The exit_code field itself is unchanged — this is purely additive context.
jooray added a commit to jooray/hermes-agent that referenced this pull request Apr 5, 2026
* upstream/main: (29 commits)
  style: use module-level re import instead of local import re as _re
  Preserve numeric credential labels in auth removal
  Honor provider reset windows in pooled credential failover
  docs: update docstring to mention Fireworks strict validation
  test: add strict API validation tests for Fireworks compatibility
  test: add test for _should_sanitize_tool_calls()
  refactor: use _should_sanitize_tool_calls in run_conversation()
  refactor: use _should_sanitize_tool_calls in _handle_max_iterations()
  refactor: use _should_sanitize_tool_calls in flush_memories()
  feat: add _should_sanitize_tool_calls() method
  test(redact): add regression tests for lowercase variable redaction (NousResearch#4367) (NousResearch#5185)
  docs(skill): claude-code v2.2 — add cheat sheet commands, env vars, rules, advanced features (NousResearch#5158)
  fix(telegram): prevent duplicate message delivery on send timeout (NousResearch#5153)
  fix: strip MEDIA: directives from streamed gateway messages (NousResearch#5152)
  docs(skill): comprehensive claude-code skill rewrite v2.0 (NousResearch#5155)
  fix(security): guard cron script against path traversal and redact output
  feat: add exit code context for common CLI tools in terminal results (NousResearch#5144)
  fix: move pre_llm_call plugin context to user message, preserve prompt cache (NousResearch#5146)
  fix: --yolo and other flags silently dropped when placed before 'chat' subcommand (NousResearch#5145)
  fix: include approval metadata in terminal tool results (NousResearch#5141)
  ...
Tommyeds pushed a commit to Tommyeds/hermes-agent that referenced this pull request Apr 12, 2026
…ousResearch#5144)

When commands like grep, diff, test, or find return non-zero exit codes
that aren't actual errors (grep 1 = no matches, diff 1 = files differ),
the model wastes turns investigating non-problems. This adds an
exit_code_meaning field to the terminal JSON result that explains
informational exit codes, so the agent can move on instead of debugging.

Covers grep/rg/ag/ack (no matches), diff (files differ), find (partial
access), test/[ (condition false), curl (timeouts, DNS, HTTP errors),
and git (context-dependent). Correctly extracts the last command from
pipelines and chains, strips full paths and env var assignments.

The exit_code field itself is unchanged — this is purely additive context.
angelburgosrosado pushed a commit to angelburgosrosado/hermes-agent that referenced this pull request Apr 27, 2026
…ousResearch#5144)

When commands like grep, diff, test, or find return non-zero exit codes
that aren't actual errors (grep 1 = no matches, diff 1 = files differ),
the model wastes turns investigating non-problems. This adds an
exit_code_meaning field to the terminal JSON result that explains
informational exit codes, so the agent can move on instead of debugging.

Covers grep/rg/ag/ack (no matches), diff (files differ), find (partial
access), test/[ (condition false), curl (timeouts, DNS, HTTP errors),
and git (context-dependent). Correctly extracts the last command from
pipelines and chains, strips full paths and env var assignments.

The exit_code field itself is unchanged — this is purely additive context.
02356abc pushed a commit to 02356abc/hermes-agent that referenced this pull request May 14, 2026
…ousResearch#5144)

When commands like grep, diff, test, or find return non-zero exit codes
that aren't actual errors (grep 1 = no matches, diff 1 = files differ),
the model wastes turns investigating non-problems. This adds an
exit_code_meaning field to the terminal JSON result that explains
informational exit codes, so the agent can move on instead of debugging.

Covers grep/rg/ag/ack (no matches), diff (files differ), find (partial
access), test/[ (condition false), curl (timeouts, DNS, HTTP errors),
and git (context-dependent). Correctly extracts the last command from
pipelines and chains, strips full paths and env var assignments.

The exit_code field itself is unchanged — this is purely additive context.
olympus-terminal pushed a commit to olympus-terminal/hermes-agent that referenced this pull request May 16, 2026
…ousResearch#5144)

When commands like grep, diff, test, or find return non-zero exit codes
that aren't actual errors (grep 1 = no matches, diff 1 = files differ),
the model wastes turns investigating non-problems. This adds an
exit_code_meaning field to the terminal JSON result that explains
informational exit codes, so the agent can move on instead of debugging.

Covers grep/rg/ag/ack (no matches), diff (files differ), find (partial
access), test/[ (condition false), curl (timeouts, DNS, HTTP errors),
and git (context-dependent). Correctly extracts the last command from
pipelines and chains, strips full paths and env var assignments.

The exit_code field itself is unchanged — this is purely additive context.
gweeteve pushed a commit to gweeteve/hermes-agent that referenced this pull request Jun 2, 2026
…ousResearch#5144)

When commands like grep, diff, test, or find return non-zero exit codes
that aren't actual errors (grep 1 = no matches, diff 1 = files differ),
the model wastes turns investigating non-problems. This adds an
exit_code_meaning field to the terminal JSON result that explains
informational exit codes, so the agent can move on instead of debugging.

Covers grep/rg/ag/ack (no matches), diff (files differ), find (partial
access), test/[ (condition false), curl (timeouts, DNS, HTTP errors),
and git (context-dependent). Correctly extracts the last command from
pipelines and chains, strips full paths and env var assignments.

The exit_code field itself is unchanged — this is purely additive context.
Egavasyug pushed a commit to Egavasyug/hermes-agent that referenced this pull request Jun 10, 2026
…ousResearch#5144)

When commands like grep, diff, test, or find return non-zero exit codes
that aren't actual errors (grep 1 = no matches, diff 1 = files differ),
the model wastes turns investigating non-problems. This adds an
exit_code_meaning field to the terminal JSON result that explains
informational exit codes, so the agent can move on instead of debugging.

Covers grep/rg/ag/ack (no matches), diff (files differ), find (partial
access), test/[ (condition false), curl (timeouts, DNS, HTTP errors),
and git (context-dependent). Correctly extracts the last command from
pipelines and chains, strips full paths and env var assignments.

The exit_code field itself is unchanged — this is purely additive context.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant