Skip to content

Add feature telemetry for CLI commands#408

Merged
robinaugh merged 10 commits intomainfrom
jason/rwx-45-feature-telemetry
Mar 11, 2026
Merged

Add feature telemetry for CLI commands#408
robinaugh merged 10 commits intomainfrom
jason/rwx-45-feature-telemetry

Conversation

@robinaugh
Copy link
Contributor

@robinaugh robinaugh commented Mar 10, 2026

Summary

  • Wires a telemetry collector into the service layer, recording events for sandbox operations (start, stop, reset, exec, sync_push, sync_pull), SSH connections/commands, and run lifecycle (initiate, complete)
  • Adds cli.command and cli.error telemetry events recorded at CLI exit
  • Fixes a race where Flush() fired the telemetry POST in a background goroutine that was killed before the request completed — now sends synchronously at exit

Test plan

  • Unit tests added for all telemetry recording across service methods (service_telemetry_test.go)
  • Unit tests for error classification (errors_test.go)
  • Manual verification that telemetry events are received by the server

@robinaugh robinaugh self-assigned this Mar 10, 2026
@robinaugh robinaugh force-pushed the jason/rwx-45-feature-telemetry branch from 177c7f4 to 790548d Compare March 10, 2026 17:34
Add TelemetryCollector to cli.Config and pass it from root.go during
initialization. Add recordTelemetry helper on Service for nil-safe event
recording. Add ErrSSH and ErrPatch sentinel errors with WrapSentinel
utility for error classification. Update test helpers to wire a collector
into all test setups with drain/find helpers.
Record sandbox.start events with reuse prop
for both reattach and new-start paths. Extend SandboxSession with
CreatedAt, LastExecAt, and ExecCount fields to power lifecycle metrics.
Set CreatedAt when persisting new sandbox sessions.
Record ssh.connect events with success/failure status in both sandbox
exec and debug flows. Record ssh.command events with exit_code and
interactive flag. Wrap SSH connection failures with ErrSSH sentinel.
Change syncChangesToSandbox and pullChangesFromSandbox signatures to
return patch bytes for telemetry tracking. Record sync_push events with
patch size, duration, and LFS skip count. Record sync_pull events with
patch size, duration, success status, and reject file count. Wrap patch
application failures with ErrPatch sentinel for error classification.
Record sandbox.exec events with overall duration, exit code, sync push/
pull timing, and total patch bytes. Update SandboxSession ExecCount and
LastExecAt after each exec to support lifecycle analytics.
Record run.initiate event before API call with flags for targeted tasks
and initialization parameters.
Record run.complete event when polling detects a completed run, with
result status, wait duration, and whether the caller was in wait mode.
Flush() was firing the HTTP POST in a background goroutine, but the CLI
exits immediately after calling Flush(), killing the goroutine before
the request completes. Make the send synchronous since it only runs at
CLI exit.
@robinaugh robinaugh force-pushed the jason/rwx-45-feature-telemetry branch from 790548d to fb1fd94 Compare March 10, 2026 18:27
@robinaugh robinaugh marked this pull request as ready for review March 10, 2026 18:39
ErrRetry = errors.New("retry")
ErrSandboxNoGitDir = errors.New("no .git directory found in sandbox. Set 'preserve-git-dir: true' on your git/clone task")
ErrSSH = errors.New("ssh error")
ErrPatch = errors.New("patch error")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be threaded through the run service as well? Looks like its only used in the sandbox service right now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah probably - for the sake of size / blast radius, I'll log this as a separate linear issue to do before shipping this

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@robinaugh robinaugh merged commit c12e051 into main Mar 11, 2026
1 check passed
@robinaugh robinaugh deleted the jason/rwx-45-feature-telemetry branch March 11, 2026 13:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants