Problem

The sandbox process writes info-level logs to /var/log/navigator.log via tracing-appender::non_blocking in append-only mode (crates/navigator-sandbox/src/main.rs:83-106). There is no log rotation, no size limit, and no cleanup mechanism.

Log volume scales linearly with network activity — one structured log line (~500-1000 bytes) per L4 CONNECT request and one per L7 HTTP request. A sandbox running a coding agent making thousands of API calls per session can produce 10-20 MB/day. Over days of continuous use, this grows unbounded and risks:

• Ephemeral storage exhaustion: Unbounded growth can exhaust the container overlay filesystem or the node's ephemeral storage, causing kubelet to evict the pod.
• Loss of diagnostic data: No rotation means no ability to ship older logs externally. When the container dies, all logs vanish.
• Silent log drops: tracing_appender::non_blocking drops events when its channel fills (default 128K). If the disk fills, the background writer blocks, the channel fills, and events are silently dropped.

Current Implementation

The path is hardcoded in three places: Rust source (main.rs), Dockerfile (deploy/docker/Dockerfile.sandbox), and the e2e test (e2e/python/test_sandbox_policy.py:_read_navigator_log()).

Proposed Solution

Switch from a raw std::fs::File to tracing_appender::rolling::RollingFileAppender with daily rotation and max_log_files(3). This is a ~10-line change with zero new dependencies (tracing-appender v0.2.4 is already in Cargo.lock).

// Replace raw file open with:
let file_appender = tracing_appender::rolling::RollingFileAppender::builder()
    .rotation(tracing_appender::rolling::Rotation::DAILY)
    .filename_prefix("navigator")
    .filename_suffix("log")
    .max_log_files(3)
    .build("/var/log")
    .into_diagnostic()?;
let (file_writer, _file_guard) = tracing_appender::non_blocking(file_appender);

Additional considerations

• E2e test update: _read_navigator_log() reads the hardcoded path /var/log/navigator.log. With rolling, files are named navigator.YYYY-MM-DD.log. The test helper needs to glob for navigator.*.log or read from the directory.
• Configurable log directory: Consider making the log directory configurable via --log-dir / NAVIGATOR_LOG_DIR (default /var/log) for testability and deployment flexibility.
• Dockerfile: The touch /var/log/navigator.log in Dockerfile.sandbox can be removed since the rolling appender creates files on demand.

Alternatives Considered

• Size-based rotation (e.g., file-rotate crate): Adds a new dependency; time-based daily rotation is sufficient for this use case.
• External logrotate: Requires cron + SIGHUP handling; adds operational complexity to a minimal container image.
• Stdout-only logging: Eliminates the file layer entirely but loses the dual-layer design (terminal-friendly warn+ on stdout, detailed info+ in file for post-hoc debugging).

Originally by @johntmyers on 2026-02-19T11:05:50.633-08:00