feat(cli): redact secrets from interactive history; add /privacy controls#1387
Conversation
…/history controls Closes #1377
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Greptile SummaryThis PR adds secret-redaction to the interactive shell history file, controlled by
Confidence Score: 5/5Safe to merge; the redaction engine and command dispatch are correct with comprehensive test coverage. The implementation is solid end-to-end: policy resolution, file pruning, pause/resume, and all three backend types are handled correctly. The bearer pattern's character class omits history_policy.py — bearer pattern character class
|
| Filename | Overview |
|---|---|
| app/cli/interactive_shell/history_policy.py | Core redaction engine and RedactingFileHistory; well-structured with good guards. Bearer token regex excludes +// (standard base64 chars), risking incomplete redaction for some opaque tokens. |
| app/cli/interactive_shell/command_registry/privacy_cmds.py | All previously flagged issues (wrong messages for FileHistory, isinstance fragility, retention=0 crash) appear resolved. Logic for /history and /privacy commands is correct. |
| app/cli/interactive_shell/history.py | Policy-based history selection is clean. clear_persisted_history leaves the live backend's _entry_count stale, causing one spurious prune on the next write after a clear. |
| app/cli/interactive_shell/session.py | Minimal change: adds prompt_history_backend field with correct TYPE_CHECKING guard. |
| app/cli/interactive_shell/loop.py | Single-line change stores the prompt_toolkit History reference on the session; correct and safe. |
| tests/cli/interactive_shell/test_history_policy.py | Comprehensive parametric tests covering all redaction patterns, retention pruning, env/config resolution, and edge cases like zero-cap and multi-line PEM blocks. |
| tests/cli/interactive_shell/test_history_commands.py | Good coverage of all /history subcommands and /privacy across all three backend types. |
Sequence Diagram
sequenceDiagram
participant User
participant PromptSession
participant RedactingFileHistory
participant HistoryFile as ~/.config/opensre/interactive_history
User->>PromptSession: types command
PromptSession->>RedactingFileHistory: store_string(raw)
alt "paused == True"
RedactingFileHistory-->>PromptSession: return (no-op)
else "paused == False"
RedactingFileHistory->>RedactingFileHistory: redact_text(raw, rules)
RedactingFileHistory->>HistoryFile: write redacted entry
RedactingFileHistory->>RedactingFileHistory: increment _entry_count
alt "_entry_count > _max_entries"
RedactingFileHistory->>HistoryFile: read + prune oldest entries
RedactingFileHistory->>HistoryFile: write pruned content
end
end
User->>PromptSession: /history off
PromptSession->>RedactingFileHistory: "backend.paused = True"
User->>PromptSession: /history clear
PromptSession->>HistoryFile: write_text("")
User->>PromptSession: /privacy
PromptSession->>RedactingFileHistory: read paused, max_entries
PromptSession-->>User: show persistence/redaction/retention table
Reviews (4): Last reviewed commit: "fix(cli): use set_max_entries for /histo..." | Re-trigger Greptile
There was a problem hiding this comment.
Pull request overview
Adds privacy hardening for the interactive shell’s persisted command history by introducing configurable secret redaction, retention controls, and new /history and /privacy UX, plus documentation and tests to support the new behavior (closes #1377).
Changes:
- Introduce
HistoryPolicy+RedactingFileHistoryto redact known secret/token shapes before writing history to disk and to enforce a retention cap. - Add
/history clear|off|on|retention <N>and/privacycommands, and wire session state so commands can toggle persistence at runtime. - Document defaults/threat model and add test coverage for redaction, retention, env/config resolution, and command behavior.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| app/cli/interactive_shell/history_policy.py | New policy + redacting/persisting history backend with retention pruning. |
| app/cli/interactive_shell/history.py | Policy-aware history backend selection; add helper to clear persisted history. |
| app/cli/interactive_shell/command_registry/privacy_cmds.py | New /history subcommands and /privacy command implementation. |
| app/cli/interactive_shell/command_registry/init.py | Registers the new privacy/history commands into the slash-command registry. |
| app/cli/interactive_shell/session.py | Stores live prompt history backend on the session for runtime mutation by commands. |
| app/cli/interactive_shell/loop.py | Passes ReplSession into prompt construction and persists backend reference. |
| app/cli/interactive_shell/config.py | Reads interactive.history config block for history policy resolution. |
| tests/cli/interactive_shell/test_history_policy.py | Tests redaction patterns, paused behavior, retention pruning, and env/file resolution. |
| tests/cli/interactive_shell/test_history_commands.py | Tests /history subcommands and /privacy output across backends. |
| tests/cli/interactive_shell/test_loop.py | Updates prompt session builder tests for the new session parameter/backend reference. |
| docs/interactive-shell-privacy.mdx | New documentation for defaults, controls, and threat model. |
| docs/docs.json | Adds the new privacy doc page to the docs navigation. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Addressed the Greptile and Copilot review items in 5cb58c4:
Also merged the latest |
5cb58c4 to
defafe5
Compare
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
|
||
| def _build_default_rules() -> tuple[RedactionRule, ...]: | ||
| raw: list[tuple[str, str, str]] = [ | ||
| ("aws_key", r"(?:AKIA|ASIA)[A-Z0-9]{16}", "[REDACTED:aws_key]"), |
There was a problem hiding this comment.
this seems very limited redacted key-rules? we should scan the entire project what other keys are used or patterns which we can add..
|
|
||
| def _build_default_rules() -> tuple[RedactionRule, ...]: | ||
| raw: list[tuple[str, str, str]] = [ | ||
| ("aws_key", r"(?:AKIA|ASIA)[A-Z0-9]{16}", "[REDACTED:aws_key]"), |
There was a problem hiding this comment.
also move this to separate constants file
|
@greptileai review |
|
rest the greptile score needs to be improved to 5/5 and pls add a demo :) |
Address Greptile review feedback to lift the score to 5/5: - P1 #1 (security): the private_key redaction pattern previously matched only the BEGIN header line, so pasted multi-line PEM blocks leaked the body and END footer to disk. Pattern now spans header through footer with a non-greedy body match. Test added that pastes a full RSA, EC, and OPENSSH PEM block and asserts both the base64 body and END marker are gone after redaction (single-string and history-store paths). - P1 #2: /history retention 0 now sets the cap by writing backend._max_entries directly and only calls _prune_to_cap when n > 0, matching the same guard that store_string already uses. Existing test_zero_sets_unlimited_without_crashing still passes. - P1 #3: /history off and /history on already distinguish the three backend cases (RedactingFileHistory, plain FileHistory, InMemoryHistory) with accurate messages, and dedicated tests cover the FileHistory branch for both subcommands. - P2 #1: backend identity check uses isinstance(backend, InMemoryHistory) instead of fragile __class__.__name__ string matching. - P2 #2: paused is initialized in __init__ with an explicit bool annotation rather than as a class-level annotation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Addressed all five Greptile inline items in 9145e4c:
Quality gate green: |
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
|
@greptileai review |
|
@yashksaini-coder did you see my reviews? |
…le (Tracer-Cloud#1377) Addresses @muddlebee's review feedback on PR Tracer-Cloud#1387: > "this seems very limited redacted key-rules? we should scan the > entire project what other keys are used or patterns which we can add" > "also move this to separate constants file" ## Changes ### Split into a dedicated constants module - New `app/cli/interactive_shell/redaction_rules.py` holds the `RedactionRule` dataclass, the `_RAW_RULES` table grouped by family, and the `redact_text` function. Patterns are now easy to audit and extend without touching the `RedactingFileHistory` plumbing. - `history_policy.py` re-exports `RedactionRule`, `DEFAULT_REDACTION_RULES`, and `redact_text` so existing imports in `history.py`, `privacy_cmds.py`, and the test suite keep working unchanged. ### Expanded ruleset (12 → 22 patterns) Scanned the codebase for credentials env-var names (44 distinct names across services + integrations) and added rules covering: **New vendor-specific patterns:** - `google_api_key` — `AIza...` (Google AI / Cloud / Firebase) - `github_oauth` — `gho_/ghu_/ghs_/ghr_` (OAuth / user-server / server-server / refresh, distinct from existing `ghp_`) - `gitlab_pat` — `glpat-...` - `openrouter_key` — `sk-or-v1-...` (with `(?!or-v1-)` lookahead in the OpenAI rule so it gets its own descriptive tag) - `huggingface_token` — `hf_...` - `stripe_restricted` — `rk_(live|test)_...` **New connection-string pattern:** - `db_url_creds` — DSN-style URLs with embedded `user:password@` for postgres/postgresql/mysql/mariadb/mongodb/mongodb+srv/redis/ amqp/amqps. Username and host stay visible (debug-friendly), only the password between `:` and `@` is replaced. **New generic env-style fallbacks** (catch the long tail — Datadog, Coralogix, Argo CD, Grafana, Honeycomb, Bitbucket, Kafka SASL, etc.): - `api_key_env` — `<NAME>_API_KEY=value` - `token_env` — `<NAME>_(TOKEN|AUTH_TOKEN|BEARER_TOKEN|BOT_TOKEN|PUBLIC_KEY)=value` - `password_env` — `<NAME>_(PASSWORD|APP_PASSWORD|SASL_PASSWORD)=value` - `secret_env` — `<NAME>_(SECRET|CLIENT_SECRET|WEBHOOK_SECRET|SIGNING_SECRET)=value` These require UPPERCASE_NAMES with explicit suffixes so they don't match inside legitimate code or comments (`api_key = config.get(...)` in Python source is preserved verbatim). ### Tests - New `tests/cli/interactive_shell/test_redaction_rules.py` with 40 cases covering each new pattern, the connection-string family, the generic env-style fallbacks, and the lowercase-Python negative case. - All 442 existing `tests/cli/interactive_shell/` tests still pass.
|
@muddlebee — addressing both review comments in
DoneMoved into a constants module. The patterns now live in Expanded ruleset 12 → 22 patterns. Scanned Vendor-specific (high-confidence prefixes):
Connection strings:
Generic env-style fallbacks (the long tail):
The generic patterns require UPPERCASE_NAMES with explicit suffixes so they don't match inside legitimate Python source — Tests
Re-triggering Greptile on this commit. Demo screenshot to follow. |
- Keep privacy and agents slash command groups in registry order - Use main loop + prompt_surface; wire prompt history backend on ReplSession - Align interactive shell tests with prompt_surface API
Avoid direct _max_entries mutation; matches RedactingFileHistory public API.
|
@greptile review |
|
@yashksaini-coder fixed conflicts and pushed some nits. |
|
🚀 Houston, we have a merge. @yashksaini-coder your PR is in orbit. Thanks for launching this one! 👋 Join us on Discord - OpenSRE : hang out, contribute, or hunt for features and issues. Everyone's welcome. |
|
@yashksaini-coder pls add proper e2e demo, missing in PR checklist |
…rols (Tracer-Cloud#1387) * feat(cli): redact secrets from interactive history; add /privacy and /history controls Closes Tracer-Cloud#1377 * fix(cli): resolve history privacy review feedback Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(cli): restore privacy commands in help Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(privacy): close greptile gaps on PEM redaction and retention 0 Address Greptile review feedback to lift the score to 5/5: - P1 #1 (security): the private_key redaction pattern previously matched only the BEGIN header line, so pasted multi-line PEM blocks leaked the body and END footer to disk. Pattern now spans header through footer with a non-greedy body match. Test added that pastes a full RSA, EC, and OPENSSH PEM block and asserts both the base64 body and END marker are gone after redaction (single-string and history-store paths). - P1 #2: /history retention 0 now sets the cap by writing backend._max_entries directly and only calls _prune_to_cap when n > 0, matching the same guard that store_string already uses. Existing test_zero_sets_unlimited_without_crashing still passes. - P1 Tracer-Cloud#3: /history off and /history on already distinguish the three backend cases (RedactingFileHistory, plain FileHistory, InMemoryHistory) with accurate messages, and dedicated tests cover the FileHistory branch for both subcommands. - P2 #1: backend identity check uses isinstance(backend, InMemoryHistory) instead of fragile __class__.__name__ string matching. - P2 #2: paused is initialized in __init__ with an explicit bool annotation rather than as a class-level annotation. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * Update app/cli/interactive_shell/history_policy.py Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> * fix(cli): use set_max_entries for /history retention Avoid direct _max_entries mutation; matches RedactingFileHistory public API. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Anwesh <8139783+muddlebee@users.noreply.github.com> Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com> Co-authored-by: muddlebee <anweshknayak@gmail.com>

Closes #1377.
Summary
The interactive shell prompt persists every typed line to
~/.config/opensre/interactive_history. Incident prompts can include tokens (AWS keys, GitHub PATs, JWTs, Bearer headers, password CLI args, PEM keys, etc.), so this PR:/history clear,/history off|on,/history retention <N>subcommands/privacyslash command that shows persistence + redaction state, retention cap, file path, and a one-line threat modeldocs/interactive-shell-privacy.mdxDefaults: redaction on, persistence on, retention cap 5000. All three are overridable via
OPENSRE_HISTORY_*env vars or theinteractive.historyblock in~/.config/opensre/config.yml.What was changed
app/cli/interactive_shell/history_policy.py—HistoryPolicy, built-inRedactionRuleset,RedactingFileHistory(subclass ofprompt_toolkit.FileHistoryoverridingstore_string)app/cli/interactive_shell/history.py— selectsInMemoryHistory/RedactingFileHistory/ rawFileHistorybased on policy; addsclear_persisted_history()helperapp/cli/interactive_shell/commands.py—_cmd_historybecomes a subcommand dispatcher, new_cmd_privacy,/privacyregistered inSLASH_COMMANDSapp/cli/interactive_shell/session.py— exposes the liveprompt_history_backendso commands can flip thepausedflag at runtimeapp/cli/interactive_shell/loop.py— passes session into_build_prompt_sessionand stores the backend referenceapp/cli/interactive_shell/config.py—read_history_settings()for the config-file tierdocs/interactive-shell-privacy.mdx(linked fromdocs/docs.jsonintegration overview group)Test plan
make test-cov— 5219 passed, 3 skipped (no regressions)make lint— cleanmake format-check— cleanmake typecheck— clean for new code (one pre-existingsentry_sdkimport error on main is unrelated)