perf: use periodic log flushing instead of per-line by jdx · Pull Request #139 · jdx/pitchfork

jdx · 2026-01-19T03:34:06Z

Summary

Replace per-line log flushing with periodic flushing (every 1 second)

Problem

The supervisor was flushing logs after every single line of stdout/stderr output. For chatty daemons that produce many log lines per second, this caused excessive I/O overhead and high CPU usage.

Solution

Remove flush() calls after each line write
Add a periodic flush interval (1 second) in the select loop
Add a final flush when the daemon exits

This maintains log durability (logs are still written to disk every second) while dramatically reducing I/O operations for high-output daemons.

Test plan

cargo build passes
Manual test with chatty daemon

🤖 Generated with Claude Code

Note

Reduces I/O by batching daemon log writes while preserving timely visibility at key events.

Remove per-line flush() after stdout/stderr writes; add tokio::time::interval(500ms) periodic flush in the select loop
Explicitly flush() before sending readiness notifications (output/HTTP/port/delay) and upon process exit; add a final flush after loop ends
Adjust event loop: rely on exit_rx.recv() for termination (remove unreachable else => break); track exit_status for final result

^{Written by Cursor Bugbot for commit e48e5c7. This will update automatically on new commits. Configure here.}

Replace per-line log flushing with periodic flushing (every 1 second). This significantly reduces I/O overhead for chatty daemons while still ensuring logs are written to disk in a timely manner. - Remove flush() calls after each stdout/stderr line - Add periodic flush interval (1 second) - Add final flush when daemon exits Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.}

cursor · 2026-01-19T03:41:04Z

src/supervisor.rs

+                        if let Err(e) = log_appender.flush().await {
+                            error!("Failed to flush log for daemon {id}: {e}");
+                        }
+                    }


Flush interval prevents fallback exit path from triggering

Low Severity

The unconditional log_flush_interval.tick() branch prevents the else => break fallback from ever executing. In tokio's select!, the else branch only fires when all other branches are disabled, but the flush interval is always active. This makes the fallback exit path at lines 768-776 (which handles "streams closed but process hasn't exited yet") unreachable. If the exit channel fails unexpectedly, the loop would run forever instead of breaking gracefully.

Additional Locations (1)

src/supervisor.rs#L757-L758

- Remove unreachable `else => break` from select! (log_flush_interval.tick() is always available, making else branch unreachable) - Reduce flush interval from 1s to 500ms for better responsiveness - Flush logs immediately before sending ready notifications so clients see logs right after `--wait-ready` returns - Add #[allow(unused_assignments)] for exit_status initialization The test_ready_http_check failure was caused by logs not being flushed before the ready notification was sent, so the client couldn't see the startup logs. The test_logs_tail_command failure was a race condition with the 1s flush interval. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

## 🤖 New release * `pitchfork-cli`: 0.3.0 -> 0.3.1 <details><summary>Changelog</summary> <blockquote> ## [0.3.1](v0.3.0...v0.3.1) - 2026-01-19 ### Added - implement daemon dependency resolution ([#135](#135)) - add restart command to CLI ([#134](#134)) ### Fixed - restart command preserves daemon dependency configuration ([#142](#142)) - add missing depends field to restart command ([#136](#136)) - set IPC socket permissions to 0600 for security ([#133](#133)) - handle shell command parsing errors instead of silently failing ([#132](#132)) ### Other - reduce unnecessary daemon cloning in loops ([#144](#144)) - use periodic log flushing instead of per-line ([#139](#139)) - refresh only tracked PIDs instead of all processes ([#141](#141)) - cache compiled regex patterns ([#143](#143)) ### Security - add rate limiting to IPC server ([#137](#137)) - canonicalize config paths to prevent symlink exploitation ([#138](#138)) - add centralized daemon ID validation ([#140](#140)) </blockquote> </details> --- This PR was generated with [release-plz](https://github.com/release-plz/release-plz/).  --- > [!NOTE] > Prepares the 0.3.1 release and updates metadata/documentation accordingly. > > - **Changelog**: Adds `0.3.1` entry detailing added dependency resolution, new `restart` command, fixes, performance tweaks, and security hardening > - **Version bumps**: Updates `version` to `0.3.1` in `Cargo.toml`, `Cargo.lock`, `docs/cli/commands.json`, `docs/cli/index.md`, and `pitchfork.usage.kdl` > - **Docs regen**: Refreshes CLI docs/spec to reflect the new version (no behavioral changes in this diff) > > Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 9f9d386. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).  --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

## 🤖 New release * `pitchfork-cli`: 1.0.0 <details><summary>Changelog</summary> <blockquote> ## [1.0.0](https://github.com/jdx/pitchfork/releases/tag/v1.0.0) - 2026-01-19 ### Added - implement daemon dependency resolution ([#135](#135)) - add restart command to CLI ([#134](#134)) ### Fixed - restart command preserves daemon dependency configuration ([#142](#142)) - add missing depends field to restart command ([#136](#136)) - set IPC socket permissions to 0600 for security ([#133](#133)) - handle shell command parsing errors instead of silently failing ([#132](#132)) ### Other - bump version to 1.0.0 ([#147](#147)) - release v0.3.1 ([#121](#121)) - reduce unnecessary daemon cloning in loops ([#144](#144)) - use periodic log flushing instead of per-line ([#139](#139)) - refresh only tracked PIDs instead of all processes ([#141](#141)) - cache compiled regex patterns ([#143](#143)) ### Security - add rate limiting to IPC server ([#137](#137)) - canonicalize config paths to prevent symlink exploitation ([#138](#138)) - add centralized daemon ID validation ([#140](#140)) </blockquote> </details> --- This PR was generated with [release-plz](https://github.com/release-plz/release-plz/).  --- > [!NOTE] > Prepares the 1.0.0 release and updates `CHANGELOG.md` with the finalized notes. > > - Adds `1.0.0` section: daemon dependency resolution, new CLI `restart` command, fixes for dependency preservation and shell parsing, secure IPC socket perms, plus performance/maintenance updates > - Documents security hardening: IPC rate limiting, config path canonicalization, centralized daemon ID validation > - Retains prior `0.3.1` notes for historical context > > Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 4182984. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).

cursor bot reviewed Jan 19, 2026

View reviewed changes

jdx merged commit 8773e56 into main Jan 19, 2026
4 checks passed

jdx deleted the perf/buffered-log-writes branch January 19, 2026 04:04

jdx mentioned this pull request Jan 19, 2026

chore: release v0.3.1 #121

Merged

jdx mentioned this pull request Jan 19, 2026

chore: release v1.0.0 #148

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

perf: use periodic log flushing instead of per-line#139

perf: use periodic log flushing instead of per-line#139
jdx merged 2 commits intomainfrom
perf/buffered-log-writes

jdx commented Jan 19, 2026 •

edited by cursor bot

Loading

Uh oh!

cursor bot left a comment

Uh oh!

cursor bot Jan 19, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

jdx commented Jan 19, 2026 • edited by cursor bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Problem

Solution

Test plan

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor bot Jan 19, 2026

Choose a reason for hiding this comment

Flush interval prevents fallback exit path from triggering

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

jdx commented Jan 19, 2026 •

edited by cursor bot

Loading