Skip to content

stream proxy output while command is running#268

Merged
pszymkowiak merged 1 commit intortk-ai:masterfrom
dragonGR:pr-proxy-stream
Mar 6, 2026
Merged

stream proxy output while command is running#268
pszymkowiak merged 1 commit intortk-ai:masterfrom
dragonGR:pr-proxy-stream

Conversation

@dragonGR
Copy link
Contributor

The rtk proxy was buffering command output until the child process finished which made long-running commands look stuck in TTY mode.

So this commit switches proxy execution from buffered Command::output() to a streamed spawn() flow with piped stdout/stderr so output is forwarded incrementally while the command is still running.

What was improved:
stream stdout and stderr in real time (chunked reads)
flush each chunk immediately for better interactive UX
keep captured output for tracking/analytics
preserve child exit status behavior (still returns the same failure code when command fails)

Ref:
#222

heAdz0r added a commit to heAdz0r/rtk that referenced this pull request Feb 28, 2026
…tk-ai#217, rtk-ai#196, rtk-ai#248, rtk-ai#211, rtk-ai#200, rtk-ai#192, rtk-ai#268)

Wave 1 (critical bugs):
- fix(registry): fi/done moved to IGNORED_EXACT — find no longer shadowed (rtk-ai#246)
- fix(playwright): f64 duration, specs[] structure, --reporter=json after subcmd (rtk-ai#193)
- fix(gh): should_passthrough_gh_view for --json/--jq/--template/--web in view_pr/issue/run (rtk-ai#217+196)

Wave 2 (reliability):
- fix(git): is_blob_show_arg — blob show passthrough without trailing-newline trim (rtk-ai#248)
- fix(find): parse_find_args with native -name/-type/-maxdepth/-iname support (rtk-ai#211)
- fix(main): graceful Clap fallback + parse_failures SQLite table + rtk gain --failures (rtk-ai#200)

Wave 3 (UX):
- feat(git): global options -C/-c/--git-dir/--work-tree/--no-pager/--no-optional-locks/--bare/--literal-pathspecs (rtk-ai#192)
- feat(proxy): streaming output via spawn()+threads instead of buffered output() (rtk-ai#268)

Tests: 1091 → 1117 (+26), 0 regressions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@pszymkowiak
Copy link
Collaborator

The PR doesn't compile — utils::status_code_error is referenced but never defined anywhere in the codebase.

Easy fix: replace the return Err(utils::status_code_error(...)) with the original pattern:
std::process::exit(status.code().unwrap_or(1));
The streaming logic itself looks correct. Just needs this one fix to build.

@dragonGR
Copy link
Contributor Author

dragonGR commented Mar 1, 2026

Hello, yes, it’s most likely due to the recent changes you pushed. I’ve pushed an update and however I force-pushed it, so I hope that won’t cause any problems.

@FlorianBruniaux
Copy link
Collaborator

Hi @dragonGR! PR #241 just merged and touched src/main.rs — could you rebase on latest master? The streaming proxy improvement looks solid and we'd like to get it in.

@dragonGR
Copy link
Contributor Author

dragonGR commented Mar 5, 2026

Bonjour @FlorianBruniaux ! I just rebased on top of it and works well ! Thanks.

@FlorianBruniaux
Copy link
Collaborator

Hey @dragonGR, FYI: PR #349 (TOML filter DSL) captures stdout via Stdio::piped for commands matched by a TOML filter. Your streaming work is complementary — for unmatched commands, behavior stays Stdio::inherit (unchanged). No code conflict, just flagging for architectural context.

@pszymkowiak
Copy link
Collaborator

Review: LGTM ✅

Verified locally — cargo fmt clean, cargo clippy clean, 609/609 tests pass.

Manual testing confirms:

  • Real-time streaming works (output appears progressively, not buffered until end)
  • Exit code preserved correctly
  • Token tracking still captured via parallel byte collection

Note: stdout/stderr interleaving order is not 100% deterministic (two independent threads), but this is a significant improvement over the
previous code which always printed all stdout before all stderr.

Also fixes #187 (output line reordering in proxy mode).

Mergeable, no conflicts (CLEAN status).

@pszymkowiak pszymkowiak merged commit 884e37e into rtk-ai:master Mar 6, 2026
2 of 3 checks passed
ahundt added a commit to ahundt/rtk that referenced this pull request Mar 11, 2026
Incorporates 52 upstream commits (v0.27.0 → v0.28.2):
- TOML filter DSL engine + 30 built-in filters (PRs rtk-ai#349, rtk-ai#351, rtk-ai#386)
- Graphite CLI support (PR rtk-ai#290)
- git commit -am/--amend fix via trailing_var_arg (PR rtk-ai#327)
- restore_double_dash for cargo (PR rtk-ai#326)
- gh -R/--repo passthrough, pr edit/comment fix (PRs rtk-ai#328, rtk-ai#332)
- docker compose subcommand filtering (PR rtk-ai#336)
- Telemetry tokens_saved + install_method (PRs rtk-ai#462, rtk-ai#469, rtk-ai#471)
- proxy streaming (PR rtk-ai#268)
- Diff limits increased (100→500 lines, 10→30 hunk lines)

Conflict resolution (5 files):
- cargo_cmd.rs: adopted upstream restore_double_dash, adapted streaming
  run_test() to use it, converted old split_at_double_dash tests
- git.rs: adopted upstream simplified Commit unit variant (fixes -am),
  adapted all commit tests to flat args API, added 6 new edge case tests
- init.rs: added TOML template generation alongside hook manifest
- main.rs: merged both upstream (gt, toml_filter, verify) and hooks-v2
  (cmd, hook, stream, pipe) modules, kept all tests from both sides
- utils.rs: kept hooks-v2 command_in_path/which_command + upstream English docs

Hook engine additions during merge:
- Added gt to hook_lookup() whitelist with 4 routing test cases

All 5 hook bug fixes from issue rtk-ai#361 preserved:
1. Streaming (stream.rs BufReader)
2. Handler coordination (parallel-merge + run_manifest_handlers on both paths)
3. Stderr deny (exit 2)
4. Routing whitelist (hook_lookup)
5. Vitest run injection

1182 tests pass (1 environment-dependent upstream test excluded).
ahundt added a commit to ahundt/rtk that referenced this pull request Mar 11, 2026
Incorporates 52 upstream commits (v0.27.0 → v0.28.2):
- TOML filter DSL engine + 30 built-in filters (PRs rtk-ai#349, rtk-ai#351, rtk-ai#386)
- Graphite CLI support (PR rtk-ai#290)
- git commit -am/--amend fix via trailing_var_arg (PR rtk-ai#327)
- restore_double_dash for cargo (PR rtk-ai#326)
- gh -R/--repo passthrough, pr edit/comment fix (PRs rtk-ai#328, rtk-ai#332)
- docker compose subcommand filtering (PR rtk-ai#336)
- Telemetry tokens_saved + install_method (PRs rtk-ai#462, rtk-ai#469, rtk-ai#471)
- proxy streaming (PR rtk-ai#268)
- Diff limits increased (100→500 lines, 10→30 hunk lines)

Conflict resolution (5 files):
- cargo_cmd.rs: adopted upstream restore_double_dash
- git.rs: adopted upstream simplified Commit variant (fixes -am),
  fixed test_git_status_not_a_repo via GIT_DIR env override
- init.rs: added TOML template generation alongside hook manifest,
  made resolve_claude_dir pub(crate) for config/mod.rs
- main.rs: merged upstream (gt, toml_filter, verify) and
  multi-platform (cmd, hook, stream, safety, gemini) modules
- utils.rs: accepted English doc comments

Recovery edits (safety integration restored after incorrect overwrite):
- hook/mod.rs: restored config::rules::try_remap(), safety::check_raw(),
  safety::check() per-command, FORMAT_PRESERVING/TRANSPARENT_SINKS
  pub(crate), basename extraction, safety-dependent tests
- discover/registry.rs: updated 3 wc tests for upstream IGNORED_PREFIXES

All hook engine + safety + gemini features preserved.
1332 tests pass, 0 failures, 5 ignored.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants