feat: add hook audit mode for verifiable rewrite metrics by FlorianBruniaux · Pull Request #151 · rtk-ai/rtk

FlorianBruniaux · 2026-02-16T20:43:47Z

Summary

Add opt-in audit logging to the RTK rewrite hook (RTK_HOOK_AUDIT=1)
New rtk hook-audit command to analyze collected metrics
Provides verifiable data on hook reliability instead of anecdotal "5400 commands, zero skips"

Context

A Reddit comment on r/ClaudeAI questioned hook reliability, citing unrelated GitHub issues. The criticism about lacking verifiable metrics was fair — this PR addresses it directly.

What it does

Hook logging (.claude/hooks/rtk-rewrite.sh):

_rtk_audit_log() function logs each invocation when RTK_HOOK_AUDIT=1
Format: timestamp | action | original_cmd | rewritten_cmd
Actions: rewrite, skip:already_rtk, skip:heredoc, skip:no_match, skip:no_deps, skip:empty
Log location: ~/.local/share/rtk/hook-audit.log (configurable via RTK_AUDIT_DIR)
Zero overhead when disabled (default)

Audit command (src/hook_audit_cmd.rs):

$ rtk hook-audit --since 0
Hook Audit (all time)
──────────────────────────────
Total invocations: 8
Rewrites:          5 (62.5%)
Skips:             3 (37.5%)
  no_match:      2
  already_rtk:   1
Top commands: git status (1), cargo test (1), cargo clippy (1)

Usage

export RTK_HOOK_AUDIT=1    # enable logging
# use Claude Code normally...
rtk hook-audit             # last 7 days
rtk hook-audit --since 0   # all time

Test plan

8 Rust unit tests (parse_line, base_command, filter_since_days, token_savings)
7 bash hook tests (rewrite, skip:already_rtk, skip:heredoc, skip:no_match, log format, no-log-when-disabled)
360 total tests pass, 0 clippy errors
Manual test: hook creates log, rtk hook-audit displays correct stats
Run with RTK_HOOK_AUDIT=1 in a real Claude Code session over a few days

🤖 Generated with Claude Code

Add opt-in audit logging to the RTK rewrite hook and a new `rtk hook-audit` command to analyze the collected metrics. This provides verifiable data on hook reliability (rewrite rate, skip reasons, top commands) instead of relying on anecdotal claims. - Add `_rtk_audit_log()` function to rtk-rewrite.sh (opt-in via RTK_HOOK_AUDIT=1) - Log each invocation: timestamp | action | original_cmd | rewritten_cmd - Actions: rewrite, skip:already_rtk, skip:heredoc, skip:no_match, skip:no_deps, skip:empty - New `src/hook_audit_cmd.rs` module with log parser and stats display - New `rtk hook-audit --since N` subcommand (default: last 7 days) - 8 Rust unit tests + 7 hook bash tests for audit mode - Make HOOK var overridable in test suite for repo-local testing Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

This PR adds an opt-in audit logging feature to the RTK rewrite hook to provide verifiable metrics on hook reliability. Users can enable logging via RTK_HOOK_AUDIT=1 and analyze the collected data using the new rtk hook-audit command. This addresses feedback about lacking verifiable data on hook performance and skip rates.

Changes:

New rtk hook-audit command to analyze hook invocation metrics (rewrites vs skips, top commands, time-based filtering)
Audit logging function in the hook script that logs timestamp, action type, original command, and rewritten command
Test coverage for audit logging functionality in the bash test suite

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
src/main.rs	Adds hook-audit subcommand registration
src/hook_audit_cmd.rs	New module implementing audit log parsing, filtering, and statistics display
.claude/hooks/rtk-rewrite.sh	Adds `_rtk_audit_log()` function and logging calls at each decision point
hooks/test-rtk-rewrite.sh	Adds 7 tests for audit logging feature and makes HOOK path configurable

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-16T20:49:24Z

hooks/test-rtk-rewrite.sh

+test_audit_log "audit: rewrite git status" \
+  "git status" \
+  "rewrite"
+
+test_audit_log "audit: skip already_rtk" \
+  "rtk git status" \
+  "skip:already_rtk"
+
+test_audit_log "audit: skip heredoc" \
+  "cat <<'EOF'
+hello
+EOF" \
+  "skip:heredoc"
+
+test_audit_log "audit: skip no_match" \
+  "echo hello world" \
+  "skip:no_match"
+
+test_audit_log "audit: rewrite cargo test" \
+  "cargo test" \
+  "rewrite"
+


Test coverage is missing for skip:empty and skip:no_deps actions. While the logic for these cases is simple, adding tests would ensure complete coverage of all audit action types documented in the PR description.

Copilot · 2026-02-16T20:49:24Z

.claude/hooks/rtk-rewrite.sh

+# --- Audit logging (opt-in via RTK_HOOK_AUDIT=1) ---
+_rtk_audit_log() {
+  if [ "${RTK_HOOK_AUDIT:-0}" != "1" ]; then return; fi
+  local action="$1" original="$2" rewritten="${3:--}"
+  local dir="${RTK_AUDIT_DIR:-${HOME}/.local/share/rtk}"
+  mkdir -p "$dir"
+  printf '%s | %s | %s | %s\n' \
+    "$(date -u +%Y-%m-%dT%H:%M:%SZ)" "$action" "$original" "$rewritten" \
+    >> "${dir}/hook-audit.log"
+}


Critical: The audit logging feature is only added to .claude/hooks/rtk-rewrite.sh but not to hooks/rtk-rewrite.sh. The embedded hook in src/init.rs uses include_str!("../hooks/rtk-rewrite.sh"), which means the audit feature won't be available to users who install RTK via rtk init. The changes must be applied to hooks/rtk-rewrite.sh instead of (or in addition to) .claude/hooks/rtk-rewrite.sh.

Copilot · 2026-02-16T20:49:24Z

src/hook_audit_cmd.rs

@@ -0,0 +1,283 @@
+use anyhow::{Context, Result};
+use std::collections::HashMap;
+use std::path::PathBuf;


Missing chrono import. The code uses chrono::Utc and chrono::Duration on line 60, but there's no import statement. Add the missing import to fix the compilation error.

Suggested change

use std::path::PathBuf;

use std::path::PathBuf;

use chrono::{Duration, Utc};

Copilot · 2026-02-16T20:49:25Z

.claude/hooks/rtk-rewrite.sh

+
 # Guards: skip silently if dependencies missing
 if ! command -v rtk &>/dev/null || ! command -v jq &>/dev/null; then
+  _rtk_audit_log "skip:no_deps" "-"


The audit log entry for skip:no_deps uses "-" as the original command because INPUT hasn't been read yet (line 25). This happens when jq is missing, so we can't parse the JSON. However, this makes the audit log misleading. Consider either: 1) reading INPUT before the dependency check (but handle jq missing case), 2) documenting this behavior, or 3) using a more descriptive placeholder like "(unknown - jq missing)" instead of "-".

Suggested change

_rtk_audit_log "skip:no_deps" "-"

_rtk_audit_log "skip:no_deps" "(unknown - dependencies missing before INPUT read)"

Rust binary replaces 204-line bash script as Claude Code PreToolUse hook. Adds rtk hook claude, rtk run -c, and Windows support via cfg!(windows). Closes rtk-ai#112 (chained commands missed). Based on updated master (70c3786) which includes: - Hook audit mode (rtk-ai#151) - Claude Code agents and skills (d8f4659) - tee raw output feature (rtk-ai#134) Migrated from feat/rust-hooks (571bd86) with conflict resolution for: - src/main.rs: Commands enum (preserved both hook audit + our hook commands) - src/init.rs: Hook registration (integrated both approaches) New files (src/cmd/ module): - mod.rs: Module declarations (10 modules, excluding safety/trash/gemini for PR 1) - hook.rs: Shared hook decision logic (21 tests, 3 safety tests removed for PR 2) - claude_hook.rs: Claude Code JSON protocol handler (18 tests) - lexer.rs: Quote-aware tokenizer (28 tests) - analysis.rs: Chain parsing and shellism detection (10 tests) - builtins.rs: cd/export/pwd/echo/true/false (8 tests) - exec.rs: Command executor with recursion guard (22 tests, safety dispatch removed for PR 2) - filters.rs: Output filter registry (5 tests) - predicates.rs: Context predicates (4 tests) - test_helpers.rs: Test utilities Modified files: - src/main.rs: Added Commands::Run, Commands::Hook, HookCommands enum, routing - src/init.rs: Changed patch_settings_json to use rtk hook claude binary command - hooks/rtk-rewrite.sh: Replaced 204-line bash script with 4-line shim (exec rtk hook claude) - Cargo.toml: Added which = 7 for PATH resolution - INSTALL.md: Added Windows installation section Windows support: - exec.rs:175-176: cfg!(windows) selects cmd /C vs sh -c for shell passthrough - predicates.rs:26: USERPROFILE fallback for Windows home directory - No bash, node, or bun dependency - rtk hook claude is a compiled Rust binary Tests: All 541 tests pass

Keep both hook-rewrite (this PR) and hook-audit (upstream rtk-ai#151) commands. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Rust binary replaces 204-line bash script as Claude Code PreToolUse hook. Adds rtk hook claude, rtk run -c, and Windows support via cfg!(windows). Closes rtk-ai#112 (chained commands missed). Based on updated master (70c3786) which includes: - Hook audit mode (rtk-ai#151) - Claude Code agents and skills (d8f4659) - tee raw output feature (rtk-ai#134) Migrated from feat/rust-hooks (571bd86) with conflict resolution for: - src/main.rs: Commands enum (preserved both hook audit + our hook commands) - src/init.rs: Hook registration (integrated both approaches) New files (src/cmd/ module): - mod.rs: Module declarations (10 modules, excluding safety/trash/gemini for PR 1) - hook.rs: Shared hook decision logic (21 tests, 3 safety tests removed for PR 2) - claude_hook.rs: Claude Code JSON protocol handler (18 tests) - lexer.rs: Quote-aware tokenizer (28 tests) - analysis.rs: Chain parsing and shellism detection (10 tests) - builtins.rs: cd/export/pwd/echo/true/false (8 tests) - exec.rs: Command executor with recursion guard (22 tests, safety dispatch removed for PR 2) - filters.rs: Output filter registry (5 tests) - predicates.rs: Context predicates (4 tests) - test_helpers.rs: Test utilities Modified files: - src/main.rs: Added Commands::Run, Commands::Hook, HookCommands enum, routing - src/init.rs: Changed patch_settings_json to use rtk hook claude binary command - hooks/rtk-rewrite.sh: Replaced 204-line bash script with 4-line shim (exec rtk hook claude) - Cargo.toml: Added which = 7 for PATH resolution - INSTALL.md: Added Windows installation section Windows support: - exec.rs:175-176: cfg!(windows) selects cmd /C vs sh -c for shell passthrough - predicates.rs:26: USERPROFILE fallback for Windows home directory - No bash, node, or bun dependency - rtk hook claude is a compiled Rust binary Tests: All 541 tests pass

Add opt-in audit logging to the RTK rewrite hook and a new `rtk hook-audit` command to analyze the collected metrics. This provides verifiable data on hook reliability (rewrite rate, skip reasons, top commands) instead of relying on anecdotal claims. - Add `_rtk_audit_log()` function to rtk-rewrite.sh (opt-in via RTK_HOOK_AUDIT=1) - Log each invocation: timestamp | action | original_cmd | rewritten_cmd - Actions: rewrite, skip:already_rtk, skip:heredoc, skip:no_match, skip:no_deps, skip:empty - New `src/hook_audit_cmd.rs` module with log parser and stats display - New `rtk hook-audit --since N` subcommand (default: last 7 days) - 8 Rust unit tests + 7 hook bash tests for audit mode - Make HOOK var overridable in test suite for repo-local testing Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

Rust binary replaces 204-line bash script as Claude Code PreToolUse hook. Adds rtk hook claude, rtk run -c, and Windows support via cfg!(windows). Closes rtk-ai#112 (chained commands missed). Based on updated master (0b0071d) which includes: - Hook audit mode (rtk-ai#151) - Claude Code agents and skills (d8b5bb0) - tee raw output feature (rtk-ai#134) Migrated from feat/rust-hooks (571bd86) with conflict resolution for: - src/main.rs: Commands enum (preserved both hook audit + our hook commands) - src/init.rs: Hook registration (integrated both approaches) New files (src/cmd/ module): - mod.rs: Module declarations (10 modules, excluding safety/trash/gemini for PR 1) - hook.rs: Shared hook decision logic (21 tests, 3 safety tests removed for PR 2) - claude_hook.rs: Claude Code JSON protocol handler (18 tests) - lexer.rs: Quote-aware tokenizer (28 tests) - analysis.rs: Chain parsing and shellism detection (10 tests) - builtins.rs: cd/export/pwd/echo/true/false (8 tests) - exec.rs: Command executor with recursion guard (22 tests, safety dispatch removed for PR 2) - filters.rs: Output filter registry (5 tests) - predicates.rs: Context predicates (4 tests) - test_helpers.rs: Test utilities Modified files: - src/main.rs: Added Commands::Run, Commands::Hook, HookCommands enum, routing - src/init.rs: Changed patch_settings_json to use rtk hook claude binary command - hooks/rtk-rewrite.sh: Replaced 204-line bash script with 4-line shim (exec rtk hook claude) - Cargo.toml: Added which = 7 for PATH resolution - INSTALL.md: Added Windows installation section Windows support: - exec.rs:175-176: cfg!(windows) selects cmd /C vs sh -c for shell passthrough - predicates.rs:26: USERPROFILE fallback for Windows home directory - No bash, node, or bun dependency - rtk hook claude is a compiled Rust binary Tests: All 541 tests pass

Upstream 0.22.2 sync (all previously missing fixes verified applied): - fix(lint): propagate linter exit code (rtk-ai#207) — CI false-green fix - feat: add rtk wc command for compact word/line/byte counts (rtk-ai#175) - fix(playwright): JSON parser (specs layer) + binary resolution (rtk-ai#215) - fix(grep): propagate rg exit codes 1/2 (rtk-ai#227) - fix(git): branch creation not swallowed by list mode (rtk-ai#194) - fix(git): support multiple -m flags in git commit (rtk-ai#202) - fix(grep): BRE \| translation + strip -r flag (rtk-ai#206) - fix(gh): smart markdown body filter for issue/pr view (rtk-ai#214) - fix(gh): gh run view --log-failed flag passthrough (rtk-ai#159) - feat(docker): docker compose support (rtk-ai#110) - feat: hook audit mode (rtk-ai#151) - feat: tee raw output to file (rtk-ai#134) Version bump: 0.21.1-fork.19 → 0.22.2-fork.1 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot AI review requested due to automatic review settings February 16, 2026 20:43

Copilot started reviewing on behalf of FlorianBruniaux February 16, 2026 20:44 View session

Copilot AI reviewed Feb 16, 2026

View reviewed changes

pszymkowiak merged commit 70c3786 into master Feb 16, 2026
8 of 10 checks passed

github-actions bot mentioned this pull request Feb 16, 2026

chore(master): release 0.20.0 #152

Merged

pszymkowiak mentioned this pull request Feb 16, 2026

refactor: migrate rtk-rewrite.sh hook from shell to Rust subcommand #154

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add hook audit mode for verifiable rewrite metrics#151

feat: add hook audit mode for verifiable rewrite metrics#151
pszymkowiak merged 1 commit intomasterfrom
feat/hook-audit-mode

FlorianBruniaux commented Feb 16, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 16, 2026

Uh oh!

Copilot AI Feb 16, 2026

Uh oh!

Copilot AI Feb 16, 2026

Uh oh!

Copilot AI Feb 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	use std::path::PathBuf;
	use std::path::PathBuf;
	use chrono::{Duration, Utc};

	_rtk_audit_log "skip:no_deps" "-"
	_rtk_audit_log "skip:no_deps" "(unknown - dependencies missing before INPUT read)"

Conversation

FlorianBruniaux commented Feb 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Context

What it does

Usage

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

FlorianBruniaux commented Feb 16, 2026 •

edited

Loading