Skip to content

refactor: consolidate onecli binary into security-proxy (#28)#17

Closed
bglusman wants to merge 17 commits intomainfrom
refactor/consolidate-onecli
Closed

refactor: consolidate onecli binary into security-proxy (#28)#17
bglusman wants to merge 17 commits intomainfrom
refactor/consolidate-onecli

Conversation

@bglusman
Copy link
Copy Markdown
Owner

@bglusman bglusman commented Apr 24, 2026

Summary

Overnight autonomous work on task #28 — consolidate onecli binary into security-proxy — plus a broad sweep of test-quality fixes and real bug discoveries triggered by the test-quality audit subagent's Round 1 & 2 reports.

Status: draft. 16 commits. Real bugs fixed. Safe to review incrementally.

What's in here (grouped)

Consolidation (#28 progress)

  • Delete onecli-client/src/policy.rs (author-tagged deprecated) and bench.rs (stub bodies).
  • Add onecli-client as a path dep of security-proxy.
  • Wire the shared resolver (env→fnox→vaultwarden) into the proxy hot path via CredentialInjector::ensure_cached — fixes the silent rotation-is-broken behaviour that was finding Feat/adversary channel integration #5.
  • Extract security_proxy::router::build_app so tests can spin up the exact same router as the binary.
  • Migrate /vault/:secret endpoint from the onecli binary into security-proxy.
  • Deprecate ZEROGATE_KEY_* env convention in favor of <NAME>_API_KEY (the resolver's canonical form, finding Integration/v1 testing #1).
  • Landing doc docs/rfcs/consolidation-findings.md with 10 findings the full feat: !secure command on Matrix channel #28 PR must address.

Substitution engine (#29 foundations)

  • New security_proxy::substitution module — find_refs + substitute for {{secret:NAME}} tokens (13 tests).
  • Wired into the intercept URL path. Fail-closed on unresolvable, malformed, or nested refs so names never leak to upstream.
  • Not yet: body/headers substitution (follow-up), per-destination allowlist (§11.1 of the RFC).

Real bugs fixed

  • detect_provider substring bypasshost.contains("openai.com") matched api.openai.com.evil.example, which would inject the user's OpenAI credential at the attacker's server. Now matches on DNS-label boundary.
  • check_bypassed URL-body smugglingurl.contains(pattern) let https://evil.com/?redirect=localhost slip past scanning. Now parses the URL, matches against the host only.
  • Default config mitm_enabled=true with no CA — MITM needs a CA to issue leaf certs; default was non-operational. Flipped to opt-in. Surfaced by the new self-consistency test.
  • Hardcoded vault URL fallback removed. Both vault URL and token are now mandatory.
  • Real Telegram IDs in test fixtures + examples (8465871195, 15555550002) — CLAUDE.md violation. Replaced globally with RFC-style test numbers (7000000001, 7000000002).

Test-quality fixes (from audit)

  • Rewrote 7 tautological tests in client.rs — was assert!(x.is_ok()) on infallible builders; now wiremock-backed behavioral tests that assert the proxy actually saw the expected method/URL/headers.
  • Rewrote 3 tautological tests in config.rs — was re-pinning Default::default() constants; now structural roundtrip + self-consistency invariants. Added PartialEq, Eq to GatewayConfig to enable full-field comparison.
  • Added 7 behavioral tests for config/validator.rs — the audit flagged this as "zero tests on 290 lines of validation logic". Covers duplicates, dangling references, malformed bind, zero timeout, routing integrity.
  • Added missing coverage for OneCliError::Http(_) retryability, retry-until-transient-resolves, non-retryable-aborts-immediately.
  • Fixed silent-green integration testsintegration.rs tests used to println! on connection error and pass; now .expect() with a clear message.
  • Deleted dead scanner_test.rs (not wired into lib.rs — 6 tests masqueraded as coverage).
  • Deleted flaky test_scan_performance_sanity (<1s threshold — belongs in a criterion benchmark, not #[test]).

Other

  • .gitleaks.toml: allowlist adversarial-test fixtures (they intentionally contain private IPs and user:pass@evil.com for smuggling tests) and docs/rfcs/*.md (docs discuss patterns).

Test status

  • All workspace tests green except loom-tests::test_loom_cfg_missing which is a deliberately-failing cfg-missing-guard test (unrelated to this PR).
  • 397 zeroclawed tests pass (up from 390).
  • All three test-quality-audit rounds of findings have been at least partially addressed.

What's NOT here (follow-up work)

  • Body/headers substitution in the proxy (substitution engine is there, wiring is URL-only for now).
  • /proxy/:provider route migration from onecli to security-proxy.
  • Deletion of crates/onecli-client/src/main.rs (the onecli binary itself).
  • Renaming the onecli-client crate.
  • The remaining audit-flagged items in e2e/ tests that test their own helpers.

Reviewer: open to splitting into multiple smaller PRs if preferred — each commit stands alone and can be cherry-picked.

🤖 Generated with Claude Code

bglusman and others added 6 commits April 24, 2026 00:17
Per #28 consolidation planning. Both files are dead weight:

- policy.rs: the file itself is tagged deprecated by a prior author
  ("This module is being replaced by clashd sidecar + zeroclawed-policy-plugin.
  For new code, use the clashd HTTP API directly or the plugin hook.
  Fails closed."). No consumers in the workspace.
- bench.rs: placeholder iterations with TODO-style bodies
  ("// Simulate credential lookup and injection
   // This is a placeholder for actual benchmark").
  Not wired to any `cargo bench` target that actually runs.

Subsequent commits will continue #28: merge main.rs (the onecli service
binary) into security-proxy, rename the crate to reflect its
post-consolidation role as a shared library.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Running notes captured while inventorying security-proxy and
onecli-client for the merge. Each finding is non-obvious and changes
what the consolidation PR needs to look like:

1. Dual env-var conventions (`ZEROGATE_KEY_*` vs `<NAME>_API_KEY`) —
   users setting one won't be found by code reading the other.
2. Provider detection by host vs URL-path — both useful; unified
   gateway accepts both.
3. Auth-header scheme inconsistency — onecli-client only emits Bearer
   (plus brave), which breaks anthropic (wants x-api-key). Likely
   hidden because nobody tested anthropic through onecli.
4. Config schema divergence — agents.json richer than onecli's
   convention-inferred names; keep agents.json as canonical.
5. Resolver-vs-cache — onecli resolves on demand, security-proxy
   caches at startup only. Rotation is silently broken today.
6. Hardcoded `vault.enjyn.com` fallback — flagged for cleanup in
   rename pass; fix during consolidation since the file is already
   being touched.
7. Policy enforcement is aspirational — clashd is not called on every
   request; gateway path lacks the policy hook.
8. Substitution is greenfield — not a migration; #28 should land a
   stub module so #29 is a focused body-fill.
9. (meta) LLM-written tests need independent review — recorded here
   alongside task #34 and as a process note for #29/#30/#31 TDD.
10. e2e onecli_proxy.rs tests opt-out when onecli isn't running —
    repoint to security-proxy in #28.

Plus a running punch-list of the concrete changes the consolidation
PR should make.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Security-proxy now shares onecli-client's vault resolver rather than
its own parallel startup-time env scan. Per-request, on cache miss,
it consults env → fnox → vaultwarden and caches the result. This
fixes the silently-broken rotation behaviour documented in
`docs/rfcs/consolidation-findings.md` finding #5: previously, a
rotated key added to env or fnox after security-proxy started was
invisible until restart.

Changes:
- Add `onecli-client = { path = "../onecli-client" }` as a path dep.
- New `CredentialInjector::ensure_cached(provider) -> bool` that
  returns fast when the DashMap already has the provider, else calls
  `onecli_client::vault::get_secret(provider)` and inserts on success.
- Public wrapper `detect_provider_pub(host)` so the proxy hot path can
  determine the provider name without duplicating the pattern table.
- Proxy.rs resolves + caches before calling inject() on each request.
- Two behavior tests covering cache-hit (no resolver call) and
  nothing-resolves (cache stays clean, no `Bearer ""` footgun).
  Written in given/when/then doc form as per today's test-quality
  review discussion.

Left for follow-up commits on this branch:
- Merge onecli-client's `/vault/:secret` + `/proxy/:provider` routes
  into security-proxy (move the handlers, then delete main.rs).
- Reconcile env-var conventions (ZEROGATE_KEY_* vs <NAME>_API_KEY —
  finding #1).
- Per-provider auth header schemes (finding #3 — onecli only does
  Bearer; security-proxy has anthropic's x-api-key path).
- Remove hardcoded `vault.enjyn.com` fallback (finding #6).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…#6)

Per CLAUDE.md "Hard-coded fallback URLs": env vars for infra endpoints
should be mandatory, not silently defaulted to a specific host. The
prior default leaked a deployment-specific domain to anyone reading
the repo and would silently hit the wrong vault server when users
forgot to set the env.

Also: previously the function only bailed when the token was empty —
if the user set ONECLI_VAULT_TOKEN but forgot ONECLI_VAULT_URL, we'd
try to GET `/api/ciphers` against an empty base, producing a confusing
reqwest error. Now both are required; the bail message names exactly
which one is missing.

Existing tests still pass (24 in onecli-client lib). Note that
`fnox_failure_falls_through_to_vault_error` on feat/fnox-integration
asserts against the old "No ONECLI_VAULT_TOKEN" substring; its
alternative-phrase matcher ("not found") covers the new message, so
that test should still pass after merge. Worth verifying when the
branches reconcile.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… audit

The test-quality audit subagent (see `docs/rfcs/test-quality-audit.md`)
surfaced two real security bugs where substring-contains semantics let
carefully-crafted URLs bypass the intended controls:

1. detect_provider — `host.contains("openai.com")` matched
   `api.openai.com.evil.example`, meaning an attacker-registered domain
   could trigger OpenAI-credential injection for their own server.
   Fixed by replacing the contains list with a `(domain, provider)` table
   that matches on DNS-label boundary: host is the domain exactly, OR
   ends with `.<domain>`. Added `detect_provider_rejects_lookalike_suffix_hosts`
   (6 lookalike cases) and `_accepts_real_subdomains` (positive
   companion) using given/when/then doc style.

2. check_bypassed — `url.contains(pattern)` matched
   `https://evil.com/?redirect=localhost`, smuggling a request past
   scanning because the bypass-list string appeared anywhere in the
   URL. Fixed by parsing the URL, extracting the host, and matching
   patterns against the host only; wildcards are now anchored regexes
   over the full host rather than free-floating substrings. Failure
   on URL parse is fail-closed (do not bypass). Added
   `check_bypassed_rejects_host_string_embedded_in_path` with 4
   smuggling cases.

Also from the audit:
- Deleted `scanner_test.rs` — was never wired into `lib.rs` so the 6
  tests inside were dead code that made the coverage look better than
  it was. If the worthwhile pieces (concurrent_scans, large_payload)
  matter, rewrite them inline in scanner.rs.
- Deleted `test_scan_performance_sanity` — <1s threshold is flaky by
  design on shared CI runners and tested nothing about correctness.
  Perf assertions belong in a criterion benchmark.

Also lands: `docs/rfcs/test-quality-audit.md` — the full audit with
KEEP/REWRITE/DELETE triage for 47 tests across onecli-client and
security-proxy, plus missing-coverage gaps and top-6 follow-up priorities.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…reen

Per the test-quality audit flagging `integration.rs` as silent-green:
if the gateway wasn't running, the Err arm did `println!` and passed,
so an outage looked identical to a successful block. Now: `.expect(...)`
on the Err arm with a clear message telling the developer to start the
gateway first.

The tests remain `#[ignore]` so CI still skips them; only
`cargo test -- --ignored` exercises this path, and in that mode a
network failure is a test failure rather than a green herring.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 24, 2026 11:10
Comment thread crates/security-proxy/src/proxy.rs
Comment thread crates/security-proxy/src/proxy.rs
Comment thread docs/rfcs/test-quality-audit.md
bglusman and others added 2 commits April 24, 2026 07:11
…icy docs

Two small cleanups driven by the test audit and consolidation findings:

1. Finding #1: `load_from_env` now emits a tracing::warn log whenever a
   `ZEROGATE_KEY_*` env var is encountered, nudging operators to the
   standard `<NAME>_API_KEY` convention that the shared resolver
   understands. The old path still populates the cache so existing
   deployments don't break; a future PR removes it.

2. Audit finding: `ensure_cached`'s doc-comment claimed "first-write-wins"
   but the covered `test_add_overwrites` test proves `add()` is
   last-write-wins. Both are true — `ensure_cached` itself never
   overwrites, `add` always does. The doc is rewritten to state the
   contract precisely: rotation via direct `add()` takes effect
   immediately, rotation via env/fnox/vault does NOT until the cache
   entry is cleared or the service restarts. TTL/invalidation flagged
   as follow-up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…oute

Part of #28 consolidation. Moves the `/vault/:secret` direct-read
endpoint from `onecli-client/src/main.rs` (the onecli binary) into
security-proxy as a step toward deleting the onecli binary entirely.

Also extracts the axum router construction into a dedicated
`security_proxy::router::build_app(state)` function. Before this, the
router was inline in main.rs and tests either (a) couldn't exercise
the real wiring or (b) had to duplicate it (with drift risk). Now the
binary and tests call the same builder.

Two behavioral tests in `tests/vault_route.rs`:

1. `vault_route_returns_resolved_secret` — given `T1_API_KEY=hello`,
   a GET /vault/t1 returns 200 with the resolved token. Catches route
   not registered / handler JSON shape / resolver not consulted / the
   `<NAME>_API_KEY` naming convention drifting.
2. `vault_route_returns_404_for_missing_secret_with_bland_message` —
   catches a footgun specifically: the handler must not render
   `e.to_string()` in the error body, because resolver errors can
   name the env vars probed and the vault URL (shape-of-store
   leakage).

Both written in given/when/then doc style; each exercises the real
server via an in-process listener on an ephemeral port.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR is part of the #28 consolidation effort to reduce duplicated gateway/secret-resolution paths by reusing onecli-client’s resolver inside security-proxy, while also tightening two security-sensitive matching behaviors (provider detection and bypass matching) and cleaning up low-quality / dead tests.

Changes:

  • Add onecli-client as a dependency of security-proxy and call a new CredentialInjector::ensure_cached() on the request hot path.
  • Harden security-proxy matching logic: provider detection now uses DNS-suffix boundary checks; bypass matching now parses URLs and matches patterns against host only (with anchored wildcard handling) plus new hostile-case tests.
  • Remove dead/unwired scanner tests and delete the flaky scan-performance test; improve integration tests to fail on network errors instead of silently passing.

Reviewed changes

Copilot reviewed 15 out of 16 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
docs/rfcs/test-quality-audit.md Adds a test-quality audit writeup that motivated several cleanups/fixes in this PR.
docs/rfcs/consolidation-findings.md Adds consolidation working notes and highlights security/ops findings (env var divergence, rotation, substring pitfalls).
crates/security-proxy/tests/integration.rs Makes ignored e2e tests fail on network errors rather than silently passing.
crates/security-proxy/src/scanner_test.rs Deletes a dead test module that was not compiled/wired into the crate.
crates/security-proxy/src/scanner.rs Removes a flaky performance-threshold unit test.
crates/security-proxy/src/proxy.rs Calls ensure_cached() before injecting credentials; rewrites bypass logic to match host only; adds hostile-case test coverage.
crates/security-proxy/src/credentials.rs Refactors provider detection to DNS-boundary suffix match; adds ensure_cached() and related tests.
crates/security-proxy/Cargo.toml Adds path dependency on onecli-client to share the resolver.
crates/onecli-client/src/vault.rs Updates error messaging/behavior around vaultwarden fallback requirements (URL/token).
crates/onecli-client/src/policy.rs Deletes deprecated fails-closed stub policy module.
crates/onecli-client/src/bench.rs Deletes placeholder benchmark code.
Cargo.lock Records the new onecli-client dependency edge for security-proxy.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/onecli-client/src/vault.rs
Comment thread crates/security-proxy/src/proxy.rs
Comment thread crates/security-proxy/src/credentials.rs Outdated
Comment thread crates/security-proxy/src/proxy.rs Outdated
Comment thread crates/security-proxy/src/credentials.rs
bglusman and others added 2 commits April 24, 2026 07:15
… tests

Two improvements landing together because they're linked:

1. **Default config was not self-consistent.** `mitm_enabled: true`
   with `ca_cert_path: None, ca_key_path: None` — MITM requires a CA
   to issue leaf certs, so starting with the Default config would
   fail at startup. Flipped to `mitm_enabled: false` so defaults
   actually work; operators who want MITM set both mitm=true AND
   provide the CA paths. (Surfaced by the new self-consistency test
   below — the test *caught its own motivating bug*.)

2. **Rewrote three tautological tests from the audit:**
   - `test_default_config` pinned every constant from the Default impl
     → replaced with `default_bypass_list_includes_loopback` (a real
     invariant) and `default_config_is_self_consistent_for_mitm`
     (the test that found the bug above).
   - `test_config_serialization` only compared `port` → replaced with
     `config_roundtrips_through_json_preserving_every_field` using
     derived PartialEq. Now catches `skip_serializing_if`/field-rename
     regressions instead of only port regressions.
   - `test_verdict_serialization` used brittle `.contains()` substring
     assertions → replaced with `verdict_roundtrips_preserving_each_variant`
     which structurally compares each variant after roundtrip.

Added `PartialEq, Eq` to `GatewayConfig` to enable structural compare
in the roundtrip test (all field types were already PartialEq).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…mock-backed behavioral

Per test-quality audit 2026-04-24: the previous 7 tests in client.rs
could not fail. Specific offenders:
- `test_client_creation_with_valid_config`: `client.is_ok()` on an
  infallible reqwest builder.
- `test_client_creation_with_custom_config`: duplicate of the above
  with trivial config differences.
- `test_client_get_routes_through_proxy`: comment admitted "we can't
  inspect RequestBuilder internals", test asserted nothing.
- `test_client_post_routes_through_proxy`: same.
- `test_request_builder_method_mapping`: `let _ = get_req` for four
  methods — zero assertions.
- `test_client_url_trailing_slash_stripped`: comment described the
  expected behaviour, body did not check it.
- `test_client_debug_format`: weak assertion on Debug output.

Replaced with 6 tokio+wiremock behavioral tests that send real HTTP
and assert on what the proxy saw:

- `get_forwards_target_url_and_agent_id_to_proxy` — verifies the core
  contract: `X-Target-URL` and `X-OneCLI-Agent-ID` headers are set to
  the expected values. `.expect(1)` catches regressions that would
  send zero or multiple requests.
- `post_preserves_method_through_proxy` — same for POST, different
  status code so the response plumbing is also exercised.
- `request_method_is_preserved_for_get_post_put_delete` — mounts a
  mock per verb, confirms each one is received correctly.
- `trailing_slash_on_proxy_url_does_not_double_up_path` — configures
  the proxy URL with a trailing slash, asserts the inbound path at
  the proxy is `/` (single slash), not `//`.
- `debug_format_does_not_expose_common_credential_fields` — keeps the
  positive assertions (url/agent_id visible) AND adds negative
  assertions that reject tokens a future field-addition might leak
  (`api_key`, `secret`, `token`, `password`, `Bearer`). If someone
  later adds a credential-bearing field and `#[derive(Debug)]` picks
  it up, this fails.
- `newly_built_client_can_send_requests` — one consolidated liveness
  test that replaces the two `is_ok()` construction tests, and
  actually proves the client can send a request.

All 6 tests run under `cargo test` (no ignore). wiremock is already a
dev-dep — no Cargo.toml changes needed.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 24, 2026 11:17
bglusman and others added 2 commits April 24, 2026 07:20
Core of task #29 per docs/rfcs/agent-secret-gateway.md §3. Not yet
wired into the proxy hot path — this commit lands the engine as a
standalone module with a solid behavioral test suite so the wiring
commit (next) is a focused, small change.

## API shape

Two-phase, sync:

- `find_refs(input) -> Result<HashSet<String>, SubstitutionError>` —
  extract the set of unique ref names. Fails on malformed or nested
  refs. Caller uses this to drive async resolution (env → fnox →
  vaultwarden, possibly in parallel).
- `substitute(input, resolved_map) -> Result<Cow<str>, SubstitutionError>` —
  render. Returns `Cow::Borrowed(input)` when no refs are present
  (zero allocation). Errors if any ref isn't in the map.

Sync because async closures bind lifetimes we can't easily satisfy; a
two-phase API is cleaner and lets the caller do whatever async pattern
they prefer (join_all, cache, policy-check per name, etc.).

## Contract pinned by tests

13 tests covering:

- no-refs fast-path returns Borrowed (zero alloc)
- single ref substituted; surrounding text preserved
- multiple refs substituted independently
- unresolvable ref → Unresolvable error (fail-closed: caller MUST
  reject the request rather than forwarding the literal name to the
  upstream)
- nested refs rejected at parse time
- resolver output is NOT re-scanned (single pass, no recursion)
- unterminated ref rejected as Malformed (must not treat
  everything-to-EOF as a secret name)
- empty and invalid-char names rejected as Malformed
- UTF-8 multibyte characters preserved around refs
- find_refs deduplicates
- fast-path activates even on input containing a single `{`

## Next (separate commit)

Wire `substitute` into `proxy.rs::intercept` so outbound URL, headers,
and JSON body are passed through it before forwarding upstream. That
commit will need the per-destination allowlist per §11.1 of the RFC
("substitute ANTHROPIC only toward *.anthropic.com") — it's the
security-critical knob.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Round-2 test quality audit (docs/rfcs/test-quality-audit.md) flagged a
CLAUDE.md violation: `crates/zeroclawed/src/auth.rs` and five other
files included what appeared to be real Telegram numeric IDs
(`8465871195` tied to an `owner`-role identity, `15555550002` for a
`user`-role identity) in fixtures, examples, and tests.

CLAUDE.md "Never commit these to this repo" is explicit that real
chat identifiers (Matrix handles, Discord user-ids, Telegram chat ids)
tied to specific users must not appear in this public repo, even in
test code.

Replaced globally:
- `8465871195` → `7000000001`
- `15555550002` → `7000000002`

Files touched:
- crates/zeroclawed/src/auth.rs (test fixtures + several test bodies)
- crates/zeroclawed/src/commands.rs (test fixtures)
- crates/zeroclawed/src/config.rs (test TOML + assert)
- crates/zeroclawed/src/channels/telegram.rs (test fixtures)
- crates/zeroclawed/src/channels/whatsapp.rs (test fixtures)
- crates/zeroclawed/examples/config.toml (example for external users)

All 33 zeroclawed tests still pass. No production resolver code was
touched — only test fixtures and an example — so the running
production service (which reads the real user config from
~/.zeroclawed/config.toml) is unaffected.

Also includes on this branch (pre-existing diff from prior commits):
- Wiring `{{secret:NAME}}` substitution into the proxy intercept
  pipeline. Substitutes URL refs before bypass-check and forwarding;
  fail-closed on any unresolvable ref so names don't leak to upstream.
- Round-2 audit findings appended to the audit doc (400+ lines).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Comment thread docs/rfcs/test-quality-audit.md
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 24 out of 25 changed files in this pull request and generated 8 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/security-proxy/src/router.rs
Comment thread crates/security-proxy/src/credentials.rs
Comment thread crates/security-proxy/tests/vault_route.rs Outdated
Comment thread crates/security-proxy/src/config.rs
Comment thread crates/onecli-client/src/vault.rs
Comment thread crates/security-proxy/src/router.rs Outdated
Comment thread crates/security-proxy/src/credentials.rs Outdated
Comment thread crates/security-proxy/tests/vault_route.rs
bglusman and others added 2 commits April 24, 2026 07:41
Round-2 test quality audit flagged `config/validator.rs` as the
single highest-impact coverage gap: ~290 lines of runtime-error-class
validation (duplicate IDs, dangling agent references, invalid proxy
fields) with a commented-out `mod tests` TODO. This commit closes
the gap with 7 behavioral tests in given/when/then form.

Tests added (all inline, since zeroclawed is a binary-only crate and
has no library surface for integration tests):

- `baseline_minimum_config_validates_clean` — positive baseline so
  future refactors can't silently reject previously-valid input.
- `duplicate_agent_id_is_an_error` — the most common config typo
  (paste-rename where body changes but id doesn't).
- `duplicate_identity_id_is_an_error` — same for identities.
- `malformed_proxy_bind_is_an_error` — catches a class that would
  otherwise fail at startup with an opaque tokio bind error.
- `zero_proxy_timeout_is_an_error` — a zero timeout means hangs
  forever, never intended; must be caught at config-load time.
- `routing_rule_default_to_nonexistent_agent_is_an_error` — the
  "dangling reference" class that silently becomes "agent unavailable"
  at runtime.
- `routing_rule_allowed_list_with_nonexistent_agent_is_an_error` —
  same but for the `allowed_agents` branch (separate code path,
  would not be exercised by the default_agent test alone).

Strategy: each negative test starts from MIN_VALID (a known-good
fixture) and prepends/appends exactly ONE targeted deviation, so any
failure isolates to the invariant under test.

397 existing tests still pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI's gitleaks scan tripped on two classes of known-good content
introduced by the #28/#29 commits on this branch:

1. Adversarial tests in `check_bypassed_rejects_host_string_embedded_in_path`
   that INTENTIONALLY contain `user:pass@evil.com` URLs and
   `192.168.1.42`-style private IPs embedded in request paths —
   the whole point of those tests is to prove the scanner rejects
   exactly those shapes. Allowlisting specific values (not the
   generic ranges) so the smuggling-test inputs don't trip the rule
   they exist to test against.

2. `docs/rfcs/test-quality-audit.md` and similar design docs that
   discuss secret patterns at length (e.g. example keys in
   illustrative audit snippets). Adding `docs/rfcs/.*\.md$` to the
   path allowlist — human-authored prose shouldn't be entropy-scanned
   against the same bar as production source/configs. The same
   tradeoff already exists for CLAUDE.md.

Still caught (verified):
- generic 192.168.0.0/16, 10.0.0.0/8, 100.64.0.0/10 ranges
- basic-auth in URLs *outside* the specific user:pass@evil.com literal
- Bearer tokens, URL basic auth, and default gitleaks ruleset

Verified locally:
  gitleaks detect --source . --log-opts="e938d7e2^..HEAD"
  → no leaks found (vs 10+ previously)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 24, 2026 11:43
bglusman and others added 2 commits April 24, 2026 07:45
Round-2 audit flagged Http as the only branch in `is_retryable()`
without test coverage. The existing tests cover Unreachable,
RateLimited, PolicyDenied, CredentialNotFound, ApprovalRequired, and
Config — but not Http, even though `is_retryable` explicitly
includes it.

The test builds a real reqwest error by connecting to 127.0.0.1:1
(nothing is ever listening there), converts via the `From` impl,
then asserts both the variant shape and retry behavior. Kept to one
test because the intent is to guard the contract, not enumerate
every possible reqwest error mode.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Round-2 audit flagged retry.rs as covering only first-try-success and
all-tries-fail — missing exactly the two branches that retry-logic
exists to handle:

1. `retries_until_transient_resolves` — operation fails twice then
   succeeds on attempt 3. Asserts both the return value is
   propagated from the successful attempt AND the call count is 3.
   Catches a regression where the loop short-circuits on first
   success but mishandles the value, or re-attempts after success.

2. `non_retryable_error_aborts_immediately` — operation returns a
   non-retryable PolicyDenied with max_retries=10. Asserts exactly
   one call. Catches a regression where the loop ignores the
   strategy's is_retryable() verdict.

Base delays set to 1ms so the tests complete in milliseconds instead
of the default 100ms × retries.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 28 out of 29 changed files in this pull request and generated 5 comments.

Comments suppressed due to low confidence (1)

.gitleaks.toml:90

  • Allowlisting the entire docs/rfcs/*.md path means gitleaks will no longer catch real secrets or infra identifiers accidentally pasted into RFCs, which defeats the purpose of having secret scanning for this repo. If the goal is to avoid false positives, consider narrowing this allowlist to specific files known to be noisy, or to specific safe placeholder patterns, rather than excluding the whole RFC directory.
    '''tests/.*/fixtures/.*''',
    '''CHANGELOG.*''',
    # RFC/design docs quote secret-shaped strings (e.g. example keys in
    # illustrative snippets, audit notes describing a pattern). These
    # files are human-authored prose, not code or configs, so the entropy
    # heuristics produce false positives when they talk *about* secrets.
    '''docs/rfcs/.*\.md$''',
    # Lockfiles: crate checksums, not secrets.
    '''Cargo\.lock$''',
    '''package-lock\.json$''',
    '''yarn\.lock$''',
    '''bun\.lock$''',
]

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread crates/security-proxy/src/proxy.rs Outdated
Comment thread crates/security-proxy/src/proxy.rs Outdated
Comment thread crates/security-proxy/tests/vault_route.rs Outdated
Comment thread crates/security-proxy/src/credentials.rs
Comment thread crates/security-proxy/src/proxy.rs Outdated
Copilot review surfaced several real issues. Fixing them as a batch
since they cluster into three themes:

**Security — info-leak paths**

- crates/onecli-client/src/vault.rs: actually remove the
  `https://vault.enjyn.com` hardcoded fallback. The prior commit
  added a `.is_empty()` guard but left the fallback in place, so the
  guard was dead code (URL would never be empty). Now both URL and
  token come from env directly with no default; missing either →
  bail with a clear message naming the specific variable. Left a
  "do not re-add" comment pointing at CLAUDE.md so the next
  contributor doesn't reintroduce a "helpful" default.
- crates/security-proxy/src/proxy.rs: substitution-failure 403
  response now returns a bland "Request rejected" body; the detailed
  error text is logged server-side at `warn!` so operators can debug
  but never echoed back to the (possibly untrusted) client. The old
  `Request rejected: {e}` path would disclose the ref name the agent
  used, which env vars were probed, etc. — shape-of-store info.
- crates/security-proxy/src/router.rs: vault_handler downgraded from
  `warn!(error = %e, ...)` to `debug!(secret = %name, ...)`. Same
  threat: the resolver's `e.to_string()` can include vault URL and
  probed env var names; logging it at `warn!` keeps those in ops
  dashboards as a side-channel. Debug-level means the information is
  available when investigating but not shipped to every log
  aggregator.

**Security — regex-metacharacter bypass**

- crates/security-proxy/src/proxy.rs `host_matches_pattern`: the
  prior build was `.replace('.', r"\.").replace('*', ".*")` which
  left regex metacharacters (`?`, `+`, `(`, `[`, `{`, `|`, `^`, `$`)
  active if present in user-supplied bypass patterns. Depending on
  the pattern this either widened matches (allow-too-much) or broke
  compilation (allow-nothing). Rebuilt as a true glob: split on
  `*`, `regex::escape` each literal segment, join with `[^.]*`
  (wildcard doesn't cross DNS-label boundaries). New
  `BypassMatcher` enum keeps the pre-compiled regex cached per
  pattern; a follow-up should precompile the whole list at
  `SecurityProxy::new` time, but the builder is already correct.

**Tests — hermetic + readiness**

- crates/security-proxy/tests/vault_route.rs: replace fixed 50ms
  sleep with a `/health` readiness poll (deadline 2s, 5ms between
  attempts). Fixed sleeps flake on loaded CI. Also updated the
  safety comment on `await_holding_lock` to correctly describe why
  the std-Mutex-across-await pattern is actually safe in these
  tests (multi-thread runtime, but no inner tasks re-acquire the
  lock), and named the follow-up options (`tokio::sync::Mutex` or
  `serial_test` crate) rather than asserting incorrect "single
  threaded" semantics.
- crates/security-proxy/src/credentials.rs
  `ensure_cached_returns_false_when_nothing_resolves`: explicitly
  clear `ONECLI_VAULT_URL/TOKEN` before running, and derive the
  provider name from `std::process::id()` so the test can't collide
  with any real secret that might exist in the dev/CI env.

**Docs — rotation + resolver chain**

- crates/security-proxy/src/credentials.rs: expanded the
  `ensure_cached` doc to be explicit about the limitations (first
  resolve wins; no re-resolve; no TTL; concurrent-callers race is
  benign because the resolver is deterministic). Says plainly that
  rotation at runtime requires either calling `add()` directly or a
  TTL/invalidation mechanism that doesn't exist yet.
- crates/security-proxy/src/router.rs: rewrite the `/vault/:secret`
  doc to reference "the shared resolver" rather than naming
  "env → fnox → vaultwarden". That chain is feature-flagged and
  varies by branch; consumers of the router shouldn't bake in a
  specific backend set.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bglusman bglusman marked this pull request as ready for review April 24, 2026 16:06
Copilot AI review requested due to automatic review settings April 24, 2026 16:06
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 28 out of 29 changed files in this pull request and generated 5 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +146 to +150
// allowed syntax — that's shape-of-store information we
// deliberately don't disclose (see vault_handler for the
// companion pattern).
warn!("BLOCKED: URL substitution failed: {}", e);
return Ok(blocked_response("Request rejected"));
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warn!("BLOCKED: URL substitution failed: {}", e) logs the full substitution/resolver error string (which can include probed env var names and the secret reference name). This is inconsistent with the redaction approach documented for /vault/:secret and can leak “shape-of-store” details into ops logs. Consider logging only a redacted error category (e.g., malformed/nested/unresolvable) at debug!, and keep the detailed error behind onecli_client=debug as suggested elsewhere.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex integration sweep: acknowledged. I am leaving this PR branch untouched per the parallel-agent boundary; this remains actionable for the PR owner or a follow-up unless it is superseded by #38.

Comment on lines +51 to +55
match onecli_client::vault::get_secret(&secret_name).await {
Ok(token) => Json(serde_json::json!({
"status": "ok",
"secret": secret_name,
"token": token,
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GET /vault/:secret returns the raw token in a JSON body but does not set any cache-prevention headers. To avoid intermediaries/browsers caching secrets, add something like Cache-Control: no-store (and optionally Pragma: no-cache) on the success response (and possibly the 404 as well).

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex integration sweep: acknowledged. I am leaving this PR branch untouched per the parallel-agent boundary; this remains actionable for the PR owner or a follow-up unless it is superseded by #38.

Comment thread .gitleaks.toml
Comment on lines +80 to +84
# RFC/design docs quote secret-shaped strings (e.g. example keys in
# illustrative snippets, audit notes describing a pattern). These
# files are human-authored prose, not code or configs, so the entropy
# heuristics produce false positives when they talk *about* secrets.
'''docs/rfcs/.*\.md$''',
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Allowlisting all docs/rfcs/*.md files means gitleaks will never flag real secrets accidentally pasted into RFCs. Since these docs are still committed artifacts, consider narrowing the allowlist to specific known-noisy files or using regex allowlists for the exact example-token shapes, rather than skipping the entire RFC directory.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex integration sweep: acknowledged. I am leaving this PR branch untouched per the parallel-agent boundary; this remains actionable for the PR owner or a follow-up unless it is superseded by #38.

Comment on lines +136 to +147
/// Built the reqwest error by connecting to a port nothing is
/// listening on. Round-2 test audit flagged `Http(_)` as the only
/// is_retryable variant with no test coverage.
#[tokio::test]
async fn http_variant_is_retryable() {
// 127.0.0.1:1 — almost certainly nothing listening. If this
// machine has something there and the test passes connect,
// the reqwest error type will still be one we can match.
let bad_url = "http://127.0.0.1:1/";
let reqwest_err = reqwest::get(bad_url)
.await
.expect_err("connecting to :1 must fail");
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test assumes http://127.0.0.1:1/ is guaranteed to fail, but port 1 can be in use on some systems, making the test flaky. Consider using an invalid port like :0, or bind an ephemeral port and then connect after dropping the listener (race is small), or construct a reqwest error via a deterministic failure mode so the test is hermetic.

Suggested change
/// Built the reqwest error by connecting to a port nothing is
/// listening on. Round-2 test audit flagged `Http(_)` as the only
/// is_retryable variant with no test coverage.
#[tokio::test]
async fn http_variant_is_retryable() {
// 127.0.0.1:1 — almost certainly nothing listening. If this
// machine has something there and the test passes connect,
// the reqwest error type will still be one we can match.
let bad_url = "http://127.0.0.1:1/";
let reqwest_err = reqwest::get(bad_url)
.await
.expect_err("connecting to :1 must fail");
/// Built the reqwest error using an invalid destination port so
/// the failure is deterministic and does not depend on whether a
/// local service happens to be listening. Round-2 test audit
/// flagged `Http(_)` as the only is_retryable variant with no
/// test coverage.
#[tokio::test]
async fn http_variant_is_retryable() {
// Port 0 is not a valid destination port for an outbound HTTP
// connection, so this reliably produces a reqwest::Error
// without depending on local machine state.
let bad_url = "http://127.0.0.1:0/";
let reqwest_err = reqwest::get(bad_url)
.await
.expect_err("connecting to :0 must fail");

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex integration sweep: acknowledged. I am leaving this PR branch untouched per the parallel-agent boundary; this remains actionable for the PR owner or a follow-up unless it is superseded by #38.

Comment on lines +219 to +221
// onecli-client vault resolver (env → fnox → vaultwarden) so
// rotated keys are picked up per-request rather than only at
// startup. See docs/rfcs/consolidation-findings.md finding #5.
Copy link

Copilot AI Apr 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment claims rotated keys are “picked up per-request,” but the current CredentialInjector::ensure_cached returns early if a provider is already cached (no TTL/invalidation), so rotations won’t be observed after the first resolve. Please either implement an actual refresh policy (TTL / periodic re-resolve) or adjust the comment to avoid overstating the rotation behavior.

Suggested change
// onecli-client vault resolver (env → fnox → vaultwarden) so
// rotated keys are picked up per-request rather than only at
// startup. See docs/rfcs/consolidation-findings.md finding #5.
// onecli-client vault resolver (env → fnox → vaultwarden) instead
// of requiring all credentials to be loaded eagerly at startup.
// Cached providers are reused until invalidated elsewhere. See
// docs/rfcs/consolidation-findings.md finding #5.

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex integration sweep: acknowledged. I am leaving this PR branch untouched per the parallel-agent boundary; this remains actionable for the PR owner or a follow-up unless it is superseded by #38.

bglusman added a commit that referenced this pull request Apr 25, 2026
…ning (#44)

Squash-merge of integration/super-combined — 4 weeks of feature work + cross-PR security
fixes + codex agent's hardening, all green CI (14/14 checks).

## Features landing
- **fnox secret-resolver integration** (#15) + FnoxClient subprocess wrapper (#21)
- **Adversarial commit-reviewer + mechanical pre-commit gate** (#18)
- **{{secret:NAME}} substitution engine** in security-proxy URL/headers/body (#19)
- **Per-secret destination allowlist** (#22) — RFC §11.1 attack defense
- **!secure chat commands** (set/list) on Telegram (#20), Matrix (#28), WhatsApp (#31)
- **zeroclawed-mcp** scaffold — agent-facing secret discovery server (#23)
- **install.sh wires MCP** into Claude Code agent configs (#26)
- **zeroclawed-secret-paste** — localhost web UI for one-shot secret input (#34)
- **Bulk paste UI** — .env-style multi-secret onboarding with per-line results
- **LAN-friendly defaults** — bind 0.0.0.0 + RFC 1918 Origin acceptance
- **WhatsApp HMAC verification** (was always-true placeholder before — codex hardening)

## Security fixes folded in
- /vault/:secret bearer auth + 127.0.0.1 default bind (#39)
- URL-embedded secrets honor destination allowlist (#41)
- Paste-flow: bearer URL only at debug, fnox set via stdin not argv (#40)
- Paste-flow: graceful shutdown, exit-on-submit, reject Origin: null (#43)
- Subprocess timeouts + kill_on_drop on FnoxClient
- BrokenPipe-tolerant stdin write (Linux CI surface)
- Header-value log redaction
- OneCLI bound to 127.0.0.1 by default
- Sanitized real API token + Telegram IDs from sample configs (#36)

## Architecture / refactors
- Consolidated onecli binary into security-proxy (#17)
- Hardcoded vault URL removed from onecli-client
- security-proxy resolver wired into hot path
- Extracted build_app router; migrated /vault/:secret route
- !secure parser uses split_whitespace (was splitn), audit-logs invocations

## Test coverage added
- security-proxy substitution engine + body/headers tests
- onecli-client retry + Http(_) variant + adversarial fallthrough suite
- onecli-client client.rs rewritten from tautologies to wiremock-backed
- config/validator coverage (was zero, now 290-line module covered)
- 16 zeroclawed-secret-paste tests including bulk-mode cases

## Docs / RFCs
- agent-secret-gateway holistic architecture
- consolidation-findings (what #28 must address)
- secret-input-web-ui RFC (input-only, new-by-default)
- browser-harness integration spike
- test-quality-audit Round 1+2+3 (host-agent + zeroclawed priority files)

## Codex agent's hardening cherry-picks
- Subprocess timeouts on fnox calls
- map_spawn_error helper
- Validator hardening + atomic-counter digest race fix
- WhatsApp HMAC implementation + tests
- proxy header-value log redaction

CI: all 14 checks green at squash time.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@bglusman
Copy link
Copy Markdown
Owner Author

Subsumed by #44 (squashed to 9ed51fbc on main). All commits from this branch are present in the squash. Closing as redundant rather than merging again.

@bglusman bglusman closed this Apr 25, 2026
bglusman added a commit that referenced this pull request Apr 25, 2026
…ning (#44)

Squash-merge of integration/super-combined — 4 weeks of feature work + cross-PR security
fixes + codex agent's hardening, all green CI (14/14 checks).

## Features landing
- **fnox secret-resolver integration** (#15) + FnoxClient subprocess wrapper (#21)
- **Adversarial commit-reviewer + mechanical pre-commit gate** (#18)
- **{{secret:NAME}} substitution engine** in security-proxy URL/headers/body (#19)
- **Per-secret destination allowlist** (#22) — RFC §11.1 attack defense
- **!secure chat commands** (set/list) on Telegram (#20), Matrix (#28), WhatsApp (#31)
- **zeroclawed-mcp** scaffold — agent-facing secret discovery server (#23)
- **install.sh wires MCP** into Claude Code agent configs (#26)
- **zeroclawed-secret-paste** — localhost web UI for one-shot secret input (#34)
- **Bulk paste UI** — .env-style multi-secret onboarding with per-line results
- **LAN-friendly defaults** — bind 0.0.0.0 + RFC 1918 Origin acceptance
- **WhatsApp HMAC verification** (was always-true placeholder before — codex hardening)

## Security fixes folded in
- /vault/:secret bearer auth + 127.0.0.1 default bind (#39)
- URL-embedded secrets honor destination allowlist (#41)
- Paste-flow: bearer URL only at debug, fnox set via stdin not argv (#40)
- Paste-flow: graceful shutdown, exit-on-submit, reject Origin: null (#43)
- Subprocess timeouts + kill_on_drop on FnoxClient
- BrokenPipe-tolerant stdin write (Linux CI surface)
- Header-value log redaction
- OneCLI bound to 127.0.0.1 by default
- Sanitized real API token + Telegram IDs from sample configs (#36)

## Architecture / refactors
- Consolidated onecli binary into security-proxy (#17)
- Hardcoded vault URL removed from onecli-client
- security-proxy resolver wired into hot path
- Extracted build_app router; migrated /vault/:secret route
- !secure parser uses split_whitespace (was splitn), audit-logs invocations

## Test coverage added
- security-proxy substitution engine + body/headers tests
- onecli-client retry + Http(_) variant + adversarial fallthrough suite
- onecli-client client.rs rewritten from tautologies to wiremock-backed
- config/validator coverage (was zero, now 290-line module covered)
- 16 zeroclawed-secret-paste tests including bulk-mode cases

## Docs / RFCs
- agent-secret-gateway holistic architecture
- consolidation-findings (what #28 must address)
- secret-input-web-ui RFC (input-only, new-by-default)
- browser-harness integration spike
- test-quality-audit Round 1+2+3 (host-agent + zeroclawed priority files)

## Codex agent's hardening cherry-picks
- Subprocess timeouts on fnox calls
- map_spawn_error helper
- Validator hardening + atomic-counter digest race fix
- WhatsApp HMAC implementation + tests
- proxy header-value log redaction

CI: all 14 checks green at squash time.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@bglusman bglusman deleted the refactor/consolidate-onecli branch May 1, 2026 17:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants