feat: tee raw output to file for LLM re-read without re-run#134
feat: tee raw output to file for LLM re-read without re-run#134pszymkowiak merged 2 commits intomasterfrom
Conversation
When RTK filters command output, LLM agents lose failure details (stack traces, assertions) and re-run the same command 2-3x. The tee feature saves raw output to ~/.local/share/rtk/tee/ on failure and prints a one-line hint so the agent can read the file instead. - Add src/tee.rs: core module with tee_raw(), tee_and_hint(), rotation - Add TeeConfig to config.rs: enabled, mode, max_files, max_file_size - Integrate in 7 modules: cargo, runner, vitest, pytest, lint, tsc, go - Default: failures only, skip <500 chars, 20 file rotation, 1MB cap - Env overrides: RTK_TEE=0 (disable), RTK_TEE_DIR (custom directory) - 14 unit tests, 352 total tests passing, zero regressions Closes #86 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CI validate-docs requires main.rs module count == ARCHITECTURE.md count. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This pull request adds a "tee" feature to RTK that saves raw unfiltered command output to files when commands fail. This addresses a significant token waste issue where LLM agents repeatedly re-run the same failing commands to get error details that were filtered out by RTK.
Changes:
- Added new
tee.rsmodule with configurable file output, rotation, and truncation - Integrated tee feature into 7 command modules (cargo, runner, vitest, pytest, lint, tsc, go)
- Extended config system with
TeeConfigstruct and environment variable overrides - Updated documentation in README.md and CLAUDE.md to explain the feature
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| src/tee.rs | New module implementing core tee functionality: file writing, rotation, sanitization, and configuration |
| src/config.rs | Added tee field to Config struct with default initialization |
| src/main.rs | Registered the new tee module |
| src/cargo_cmd.rs | Integrated tee in run_cargo_filtered() covering all cargo subcommands |
| src/runner.rs | Integrated tee in run_err() and run_test() helper functions |
| src/vitest_cmd.rs | Integrated tee for vitest test runner output |
| src/pytest_cmd.rs | Integrated tee for pytest test runner output |
| src/lint_cmd.rs | Integrated tee for linter command output |
| src/tsc_cmd.rs | Integrated tee for TypeScript compiler output |
| src/go_cmd.rs | Integrated tee in run_test(), run_build(), and run_vet() |
| README.md | Documented tee feature configuration and usage |
| CLAUDE.md | Added tee.rs to architecture documentation |
| src/gain.rs | Unrelated formatting change to improve code readability |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Truncate at max_file_size | ||
| let content = if raw.len() > max_file_size { | ||
| format!( | ||
| "{}\n\n--- truncated at {} bytes ---", | ||
| &raw[..max_file_size], |
There was a problem hiding this comment.
The string slicing &raw[..max_file_size] on line 125 can panic if the slice boundary falls in the middle of a multi-byte UTF-8 character. This could happen when truncating command output containing Unicode characters (e.g., emoji in test output, special characters in paths).
Use a safe truncation method that respects UTF-8 character boundaries. You can use str::char_indices() to find a safe boundary or use chars().take(n).collect(). For example:
let content = if raw.len() > max_file_size {
let truncate_at = raw.char_indices()
.take_while(|(idx, _)| *idx < max_file_size)
.last()
.map(|(idx, ch)| idx + ch.len_utf8())
.unwrap_or(0);
format!(
"{}\n\n--- truncated at {} bytes ---",
&raw[..truncate_at],
max_file_size
)
} else {
raw.to_string()
};| // Truncate at max_file_size | |
| let content = if raw.len() > max_file_size { | |
| format!( | |
| "{}\n\n--- truncated at {} bytes ---", | |
| &raw[..max_file_size], | |
| // Truncate at max_file_size, respecting UTF-8 character boundaries | |
| let content = if raw.len() > max_file_size { | |
| let truncate_at = raw | |
| .char_indices() | |
| .take_while(|(idx, _)| *idx < max_file_size) | |
| .last() | |
| .map(|(idx, ch)| idx + ch.len_utf8()) | |
| .unwrap_or(0); | |
| format!( | |
| "{}\n\n--- truncated at {} bytes ---", | |
| &raw[..truncate_at], |
| if sanitized.len() > 40 { | ||
| sanitized[..40].to_string() |
There was a problem hiding this comment.
The string slicing sanitized[..40] on line 28 could theoretically panic if a non-ASCII character somehow gets through the sanitization. While the current sanitization logic replaces all non-ASCII characters with underscores (which are single-byte), this creates a fragile dependency between the two parts of the code.
For defensive programming and consistency with the codebase pattern (see utils::truncate), consider using character-based truncation:
if sanitized.chars().count() > 40 {
sanitized.chars().take(40).collect()
} else {
sanitized
}This makes the code more resilient to future changes in the sanitization logic.
| if sanitized.len() > 40 { | |
| sanitized[..40].to_string() | |
| if sanitized.chars().count() > 40 { | |
| sanitized.chars().take(40).collect() |
| let epoch = std::time::SystemTime::now() | ||
| .duration_since(std::time::UNIX_EPOCH) | ||
| .ok()? | ||
| .as_secs(); | ||
| let filename = format!("{}_{}.log", epoch, slug); |
There was a problem hiding this comment.
The filename format {epoch}_{slug}.log could lead to file overwrites if two commands with the same slug complete within the same second (same epoch timestamp). While rare, this could happen when running the same command multiple times quickly (e.g., in a test script or CI pipeline).
Consider adding nanosecond precision or a random suffix to ensure uniqueness:
let nanos = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.ok()?
.as_nanos();
let filename = format!("{}_{}.log", nanos, slug);Or use a shorter approach with subsecond precision:
let micros = epoch * 1_000_000 + (duration.subsec_micros() as u64);
let filename = format!("{}_{}.log", micros, slug);| let epoch = std::time::SystemTime::now() | |
| .duration_since(std::time::UNIX_EPOCH) | |
| .ok()? | |
| .as_secs(); | |
| let filename = format!("{}_{}.log", epoch, slug); | |
| let duration = std::time::SystemTime::now() | |
| .duration_since(std::time::UNIX_EPOCH) | |
| .ok()?; | |
| let micros = duration.as_secs() | |
| .saturating_mul(1_000_000) | |
| + duration.subsec_micros() as u64; | |
| let filename = format!("{}_{}.log", micros, slug); |
| let table_width = 3 | ||
| + 2 | ||
| + cmd_width | ||
| + 2 | ||
| + count_width | ||
| + 2 | ||
| + saved_width | ||
| + 2 | ||
| + 6 | ||
| + 2 | ||
| + time_width | ||
| + 2 | ||
| + impact_width; |
There was a problem hiding this comment.
This formatting change to break the arithmetic expression across multiple lines is unrelated to the tee feature and is not mentioned in the PR description. While the change improves readability, it should either be:
- Documented in the PR description as an incidental formatting improvement, or
- Moved to a separate PR focused on code style/formatting
Including unrelated changes in a feature PR makes code review more difficult and can obscure the actual feature changes.
|
Tee seems like a cool and useful feature. It might also help to be configurable and enabled / disabled and the destination configurable. It might also in the long run make sense to have there be a daemon that manages "sessions" and provides output uuids for saved full responses that can be re-queried, and automatically discarded based on the CLI policy. Or maybe the SQLite database could store such info. This may be worth some thought about how to revise it for better integration. I might have missed some of the implementation details based on the summary though! |
|
Thanks for the feedback! Good news — most of what you describe is already implemented in this PR: Already done:
Future ideas you raised:
The current design prioritizes zero-overhead simplicity (file write + hint line, ~25 tokens overhead) while solving the immediate re-run problem. The daemon/SQLite ideas make sense as a v2 if usage patterns justify it. Full implementation details in |
Rust binary replaces 204-line bash script as Claude Code PreToolUse hook. Adds rtk hook claude, rtk run -c, and Windows support via cfg!(windows). Closes rtk-ai#112 (chained commands missed). Based on updated master (70c3786) which includes: - Hook audit mode (rtk-ai#151) - Claude Code agents and skills (d8f4659) - tee raw output feature (rtk-ai#134) Migrated from feat/rust-hooks (571bd86) with conflict resolution for: - src/main.rs: Commands enum (preserved both hook audit + our hook commands) - src/init.rs: Hook registration (integrated both approaches) New files (src/cmd/ module): - mod.rs: Module declarations (10 modules, excluding safety/trash/gemini for PR 1) - hook.rs: Shared hook decision logic (21 tests, 3 safety tests removed for PR 2) - claude_hook.rs: Claude Code JSON protocol handler (18 tests) - lexer.rs: Quote-aware tokenizer (28 tests) - analysis.rs: Chain parsing and shellism detection (10 tests) - builtins.rs: cd/export/pwd/echo/true/false (8 tests) - exec.rs: Command executor with recursion guard (22 tests, safety dispatch removed for PR 2) - filters.rs: Output filter registry (5 tests) - predicates.rs: Context predicates (4 tests) - test_helpers.rs: Test utilities Modified files: - src/main.rs: Added Commands::Run, Commands::Hook, HookCommands enum, routing - src/init.rs: Changed patch_settings_json to use rtk hook claude binary command - hooks/rtk-rewrite.sh: Replaced 204-line bash script with 4-line shim (exec rtk hook claude) - Cargo.toml: Added which = 7 for PATH resolution - INSTALL.md: Added Windows installation section Windows support: - exec.rs:175-176: cfg!(windows) selects cmd /C vs sh -c for shell passthrough - predicates.rs:26: USERPROFILE fallback for Windows home directory - No bash, node, or bun dependency - rtk hook claude is a compiled Rust binary Tests: All 541 tests pass
Rust binary replaces 204-line bash script as Claude Code PreToolUse hook. Adds rtk hook claude, rtk run -c, and Windows support via cfg!(windows). Closes rtk-ai#112 (chained commands missed). Based on updated master (70c3786) which includes: - Hook audit mode (rtk-ai#151) - Claude Code agents and skills (d8f4659) - tee raw output feature (rtk-ai#134) Migrated from feat/rust-hooks (571bd86) with conflict resolution for: - src/main.rs: Commands enum (preserved both hook audit + our hook commands) - src/init.rs: Hook registration (integrated both approaches) New files (src/cmd/ module): - mod.rs: Module declarations (10 modules, excluding safety/trash/gemini for PR 1) - hook.rs: Shared hook decision logic (21 tests, 3 safety tests removed for PR 2) - claude_hook.rs: Claude Code JSON protocol handler (18 tests) - lexer.rs: Quote-aware tokenizer (28 tests) - analysis.rs: Chain parsing and shellism detection (10 tests) - builtins.rs: cd/export/pwd/echo/true/false (8 tests) - exec.rs: Command executor with recursion guard (22 tests, safety dispatch removed for PR 2) - filters.rs: Output filter registry (5 tests) - predicates.rs: Context predicates (4 tests) - test_helpers.rs: Test utilities Modified files: - src/main.rs: Added Commands::Run, Commands::Hook, HookCommands enum, routing - src/init.rs: Changed patch_settings_json to use rtk hook claude binary command - hooks/rtk-rewrite.sh: Replaced 204-line bash script with 4-line shim (exec rtk hook claude) - Cargo.toml: Added which = 7 for PATH resolution - INSTALL.md: Added Windows installation section Windows support: - exec.rs:175-176: cfg!(windows) selects cmd /C vs sh -c for shell passthrough - predicates.rs:26: USERPROFILE fallback for Windows home directory - No bash, node, or bun dependency - rtk hook claude is a compiled Rust binary Tests: All 541 tests pass
Rust binary replaces 204-line bash script as Claude Code PreToolUse hook. Adds rtk hook claude, rtk run -c, and Windows support via cfg!(windows). Closes rtk-ai#112 (chained commands missed). Based on updated master (70c3786) which includes: - Hook audit mode (rtk-ai#151) - Claude Code agents and skills (d8f4659) - tee raw output feature (rtk-ai#134) Migrated from feat/rust-hooks (571bd86) with conflict resolution for: - src/main.rs: Commands enum (preserved both hook audit + our hook commands) - src/init.rs: Hook registration (integrated both approaches) New files (src/cmd/ module): - mod.rs: Module declarations (10 modules, excluding safety/trash/gemini for PR 1) - hook.rs: Shared hook decision logic (21 tests, 3 safety tests removed for PR 2) - claude_hook.rs: Claude Code JSON protocol handler (18 tests) - lexer.rs: Quote-aware tokenizer (28 tests) - analysis.rs: Chain parsing and shellism detection (10 tests) - builtins.rs: cd/export/pwd/echo/true/false (8 tests) - exec.rs: Command executor with recursion guard (22 tests, safety dispatch removed for PR 2) - filters.rs: Output filter registry (5 tests) - predicates.rs: Context predicates (4 tests) - test_helpers.rs: Test utilities Modified files: - src/main.rs: Added Commands::Run, Commands::Hook, HookCommands enum, routing - src/init.rs: Changed patch_settings_json to use rtk hook claude binary command - hooks/rtk-rewrite.sh: Replaced 204-line bash script with 4-line shim (exec rtk hook claude) - Cargo.toml: Added which = 7 for PATH resolution - INSTALL.md: Added Windows installation section Windows support: - exec.rs:175-176: cfg!(windows) selects cmd /C vs sh -c for shell passthrough - predicates.rs:26: USERPROFILE fallback for Windows home directory - No bash, node, or bun dependency - rtk hook claude is a compiled Rust binary Tests: All 541 tests pass
Rust binary replaces 204-line bash script as Claude Code PreToolUse hook. Adds rtk hook claude, rtk run -c, and Windows support via cfg!(windows). Closes rtk-ai#112 (chained commands missed). Based on updated master (70c3786) which includes: - Hook audit mode (rtk-ai#151) - Claude Code agents and skills (d8f4659) - tee raw output feature (rtk-ai#134) Migrated from feat/rust-hooks (571bd86) with conflict resolution for: - src/main.rs: Commands enum (preserved both hook audit + our hook commands) - src/init.rs: Hook registration (integrated both approaches) New files (src/cmd/ module): - mod.rs: Module declarations (10 modules, excluding safety/trash/gemini for PR 1) - hook.rs: Shared hook decision logic (21 tests, 3 safety tests removed for PR 2) - claude_hook.rs: Claude Code JSON protocol handler (18 tests) - lexer.rs: Quote-aware tokenizer (28 tests) - analysis.rs: Chain parsing and shellism detection (10 tests) - builtins.rs: cd/export/pwd/echo/true/false (8 tests) - exec.rs: Command executor with recursion guard (22 tests, safety dispatch removed for PR 2) - filters.rs: Output filter registry (5 tests) - predicates.rs: Context predicates (4 tests) - test_helpers.rs: Test utilities Modified files: - src/main.rs: Added Commands::Run, Commands::Hook, HookCommands enum, routing - src/init.rs: Changed patch_settings_json to use rtk hook claude binary command - hooks/rtk-rewrite.sh: Replaced 204-line bash script with 4-line shim (exec rtk hook claude) - Cargo.toml: Added which = 7 for PATH resolution - INSTALL.md: Added Windows installation section Windows support: - exec.rs:175-176: cfg!(windows) selects cmd /C vs sh -c for shell passthrough - predicates.rs:26: USERPROFILE fallback for Windows home directory - No bash, node, or bun dependency - rtk hook claude is a compiled Rust binary Tests: All 541 tests pass
* feat: tee raw output to file for LLM re-read without re-run (rtk-ai#86) When RTK filters command output, LLM agents lose failure details (stack traces, assertions) and re-run the same command 2-3x. The tee feature saves raw output to ~/.local/share/rtk/tee/ on failure and prints a one-line hint so the agent can read the file instead. - Add src/tee.rs: core module with tee_raw(), tee_and_hint(), rotation - Add TeeConfig to config.rs: enabled, mode, max_files, max_file_size - Integrate in 7 modules: cargo, runner, vitest, pytest, lint, tsc, go - Default: failures only, skip <500 chars, 20 file rotation, 1MB cap - Env overrides: RTK_TEE=0 (disable), RTK_TEE_DIR (custom directory) - 14 unit tests, 352 total tests passing, zero regressions Closes rtk-ai#86 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix: add tee.rs to ARCHITECTURE.md module count (47 modules) CI validate-docs requires main.rs module count == ARCHITECTURE.md count. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Rust binary replaces 204-line bash script as Claude Code PreToolUse hook. Adds rtk hook claude, rtk run -c, and Windows support via cfg!(windows). Closes rtk-ai#112 (chained commands missed). Based on updated master (0b0071d) which includes: - Hook audit mode (rtk-ai#151) - Claude Code agents and skills (d8b5bb0) - tee raw output feature (rtk-ai#134) Migrated from feat/rust-hooks (571bd86) with conflict resolution for: - src/main.rs: Commands enum (preserved both hook audit + our hook commands) - src/init.rs: Hook registration (integrated both approaches) New files (src/cmd/ module): - mod.rs: Module declarations (10 modules, excluding safety/trash/gemini for PR 1) - hook.rs: Shared hook decision logic (21 tests, 3 safety tests removed for PR 2) - claude_hook.rs: Claude Code JSON protocol handler (18 tests) - lexer.rs: Quote-aware tokenizer (28 tests) - analysis.rs: Chain parsing and shellism detection (10 tests) - builtins.rs: cd/export/pwd/echo/true/false (8 tests) - exec.rs: Command executor with recursion guard (22 tests, safety dispatch removed for PR 2) - filters.rs: Output filter registry (5 tests) - predicates.rs: Context predicates (4 tests) - test_helpers.rs: Test utilities Modified files: - src/main.rs: Added Commands::Run, Commands::Hook, HookCommands enum, routing - src/init.rs: Changed patch_settings_json to use rtk hook claude binary command - hooks/rtk-rewrite.sh: Replaced 204-line bash script with 4-line shim (exec rtk hook claude) - Cargo.toml: Added which = 7 for PATH resolution - INSTALL.md: Added Windows installation section Windows support: - exec.rs:175-176: cfg!(windows) selects cmd /C vs sh -c for shell passthrough - predicates.rs:26: USERPROFILE fallback for Windows home directory - No bash, node, or bun dependency - rtk hook claude is a compiled Rust binary Tests: All 541 tests pass
Upstream 0.22.2 sync (all previously missing fixes verified applied): - fix(lint): propagate linter exit code (rtk-ai#207) — CI false-green fix - feat: add rtk wc command for compact word/line/byte counts (rtk-ai#175) - fix(playwright): JSON parser (specs layer) + binary resolution (rtk-ai#215) - fix(grep): propagate rg exit codes 1/2 (rtk-ai#227) - fix(git): branch creation not swallowed by list mode (rtk-ai#194) - fix(git): support multiple -m flags in git commit (rtk-ai#202) - fix(grep): BRE \| translation + strip -r flag (rtk-ai#206) - fix(gh): smart markdown body filter for issue/pr view (rtk-ai#214) - fix(gh): gh run view --log-failed flag passthrough (rtk-ai#159) - feat(docker): docker compose support (rtk-ai#110) - feat: hook audit mode (rtk-ai#151) - feat: tee raw output to file (rtk-ai#134) Version bump: 0.21.1-fork.19 → 0.22.2-fork.1 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Summary
Closes #86
~/.local/share/rtk/tee/on failure, print a one-line hint[full output: ~/.local/share/rtk/tee/...]so the agent reads the file instead of re-runningChanges
src/tee.rstee_raw(),tee_and_hint(), sanitization, rotation, truncationsrc/config.rsTeeConfigstruct (enabled, mode, max_files, max_file_size, directory)src/main.rsmod teesrc/cargo_cmd.rsrun_cargo_filtered()(covers build/test/clippy/check/install/nextest)src/runner.rsrun_err()andrun_test()src/vitest_cmd.rsvitest_runsrc/pytest_cmd.rspytestsrc/lint_cmd.rslintsrc/tsc_cmd.rstscsrc/go_cmd.rsrun_test(),run_build(),run_vet()README.mdCLAUDE.mdDesign decisions
.ok()?): Tee failure never affects command output or exit codetee_and_hint()helperTest plan
src/tee.rs(sanitize, should_tee, write, truncation, cleanup, config, serde)cargo fmt --all --checkcleancargo clippy --all-targetszero new warningsrtk cargo test -- nonexistentshould produce tee file + hint🤖 Generated with Claude Code