Skip to content

Add scoped and user secret providers with system key isolation#4229

Merged
amirejaz merged 7 commits intomainfrom
scoped-secret-providers
Mar 30, 2026
Merged

Add scoped and user secret providers with system key isolation#4229
amirejaz merged 7 commits intomainfrom
scoped-secret-providers

Conversation

@amirejaz
Copy link
Copy Markdown
Contributor

@amirejaz amirejaz commented Mar 19, 2026

Summary

ToolHive's secret store currently uses a flat keyspace shared between system-managed secrets (registry OAuth tokens, workload auth credentials) and user-managed secrets. This means system tokens are visible and modifiable via thv secret commands, and a Cleanup() call from one context could inadvertently wipe secrets owned by another. This PR introduces the foundational isolation layer to fix that.

  • Adds ScopedProvider: wraps any Provider and namespaces all operations under __thv_<scope>_<name>. Internal callers (registry auth, workload auth, enterprise login) will use this to keep their secrets isolated.
  • Adds UserProvider: wraps any Provider and blocks access to system-reserved keys (__thv_*). All user-facing entry points (CLI, API, MCP tool server) will use this once callers are wired up (Phase 4, Scoped secret store: wire callers to use ScopedProvider and UserProvider #4227).
  • Adds SystemKeyPrefix = "__thv_", named scopes (ScopeRegistry, ScopeWorkloads, ScopeAuth), and ErrReservedKeyName for consistent rejection of reserved key access.
  • Adds BulkDeleteSecrets to the Provider interface so both wrappers' Cleanup methods operate atomically — no-op on read-only providers (1Password, env), single write on EncryptedManager.

Closes #4224

Implementation plan

This is Phase 1 of a 5-phase rollout tracked in #4192:

Phase Issue Status
1 — Core providers + BulkDeleteSecrets #4224 This PR
2 — Factory helpers #4225 Up next
3 — Migration infrastructure #4226 Depends on Phase 2
4 — Wire callers (ships with Phase 3) #4227 Depends on Phase 3
5 — Admin escape hatch (--system flag) #4228 Depends on Phase 4

Phases 3 and 4 must ship together — wiring callers before migration runs (or vice versa) would break existing users with secrets stored under bare keys.

Type of change

  • New feature

Test plan

  • Unit tests (task test)
  • Linting (task lint-fix)

Changes

File Change
pkg/secrets/scoped.go New — ScopedProvider, UserProvider, constants, ErrReservedKeyName
pkg/secrets/scoped_test.go New — full table-driven tests for both providers
pkg/secrets/types.go Add BulkDeleteSecrets to Provider interface
pkg/secrets/encrypted.go Implement BulkDeleteSecrets (single lock, single write)
pkg/secrets/environment.go BulkDeleteSecrets no-op (read-only)
pkg/secrets/1password.go BulkDeleteSecrets no-op (read-only)
pkg/secrets/fallback.go BulkDeleteSecrets delegates to primary
pkg/secrets/mocks/mock_provider.go Regenerated
pkg/environment/environment_test.go Updated hand-written mock to satisfy interface

Does this introduce a user-facing change?

No. The wrappers exist but no callers use them yet — that happens in Phase 4 (#4227).

Migration note (Phase 4): When UserProvider is wired into user-facing code paths, any user whose secrets happen to be named with the __thv_ prefix will receive ErrReservedKeyName instead of accessing them. This prefix was previously undocumented and unprotected, so real-world conflicts are unlikely, but Phase 4 release notes should explicitly call this out.

Special notes for reviewers

The key prefix __thv_<scope>_<name> (e.g. __thv_registry_mytoken) uses double-underscore and no slashes deliberately — 1Password treats / as a path separator which caused conflicts with an earlier thv_sys/<scope>/ design. See the RFC (THV-0056) for the full discussion.

BulkDeleteSecrets is included here rather than Phase 2 because both Cleanup() implementations on ScopedProvider and UserProvider depend on it. Adding it to the interface now avoids a follow-up that would touch every provider file again.

Large PR Justification

This PR is flagged as large due to tests, but the production code change is well within limits (~265 lines across 6 files excluding tests and generated code). The size cannot be reduced further because:

Generated with Claude Code

@github-actions github-actions bot added the size/L Large PR: 600-999 lines changed label Mar 19, 2026
@github-actions github-actions bot added size/L Large PR: 600-999 lines changed and removed size/L Large PR: 600-999 lines changed labels Mar 19, 2026
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 19, 2026

Codecov Report

❌ Patch coverage is 90.82569% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.52%. Comparing base (a771117) to head (9fc1306).
⚠️ Report is 28 commits behind head on main.

Files with missing lines Patch % Lines
pkg/secrets/encrypted.go 66.66% 2 Missing and 2 partials ⚠️
pkg/secrets/1password.go 0.00% 2 Missing ⚠️
pkg/secrets/environment.go 0.00% 2 Missing ⚠️
pkg/secrets/fallback.go 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4229      +/-   ##
==========================================
+ Coverage   69.51%   69.52%   +0.01%     
==========================================
  Files         485      487       +2     
  Lines       49814    50138     +324     
==========================================
+ Hits        34627    34859     +232     
- Misses      12503    12593      +90     
- Partials     2684     2686       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions github-actions bot added size/XL Extra large PR: 1000+ lines changed and removed size/L Large PR: 600-999 lines changed labels Mar 19, 2026
Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Large PR Detected

This PR exceeds 1000 lines of changes and requires justification before it can be reviewed.

How to unblock this PR:

Add a section to your PR description with the following format:

## Large PR Justification

[Explain why this PR must be large, such as:]
- Generated code that cannot be split
- Large refactoring that must be atomic
- Multiple related changes that would break if separated
- Migration or data transformation

Alternative:

Consider splitting this PR into smaller, focused changes (< 1000 lines each) for easier review and reduced risk.

See our Contributing Guidelines for more details.


This review will be automatically dismissed once you add the justification section.

amirejaz and others added 4 commits March 26, 2026 15:39
Introduces ScopedProvider and UserProvider wrappers that isolate
system-managed secrets (registry tokens, workload auth, enterprise
login) from user-managed secrets using a reserved __thv_<scope>_
key prefix.

Also adds BulkDeleteSecrets to the Provider interface so Cleanup
operations on both wrappers are handled atomically in a single write
on EncryptedManager, consistent with how other lifecycle operations
work across read-only providers (no-op) and writable providers.

Part of #4192.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add t.Parallel() inside all t.Run closures and move ctx into
each subtest to satisfy the paralleltest and tparallel linters.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@amirejaz amirejaz force-pushed the scoped-secret-providers branch from e52d21b to e13fd99 Compare March 26, 2026 10:46
@github-actions github-actions bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Mar 26, 2026
@github-actions github-actions bot dismissed their stale review March 26, 2026 10:47

Large PR justification has been provided. Thank you!

@github-actions
Copy link
Copy Markdown
Contributor

✅ Large PR justification has been provided. The size review has been dismissed and this PR can now proceed with normal review.

- Add SecretScope typed string for scope constants (ScopeRegistry,
  ScopeWorkloads, ScopeAuth) so callers get compile-time safety
- Rename inner → provider in ScopedProvider and UserProvider structs
- Replace package-level scopedKey() with getScopedKey()/getScopePrefix()
  methods on ScopedProvider for better encapsulation
- Rename BulkDeleteSecrets → DeleteSecrets across all providers,
  tests, and mocks to better reflect the operation semantics
- Include operation name (get/set/delete) in ErrReservedKeyName error
  messages for clearer diagnostics
- Split TestEncryptedManager_BulkDeleteSecrets into four focused
  parallel tests (deletesSpecifiedKeys, persistsToDisk,
  emptyListIsNoop, nonExistentKeysNoError)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@github-actions github-actions bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Mar 27, 2026
@github-actions github-actions bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Mar 27, 2026
@amirejaz amirejaz requested a review from reyortiz3 March 27, 2026 12:00
aponcedeleonch
aponcedeleonch previously approved these changes Mar 30, 2026
Copy link
Copy Markdown
Member

@aponcedeleonch aponcedeleonch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice foundational work — the two-wrapper approach (ScopedProvider for internal callers, UserProvider for user-facing callers) is clean and the test coverage is thorough. Approving with a few comments around documenting invariants, migration safety, and batch-delete semantics.

@github-actions github-actions bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Mar 30, 2026
…ests

- Add invariant doc comment to SecretScope explaining non-empty and
  no-underscore constraints with rationale
- Add TestSecretScopeInvariants validating all declared scope constants
- Clarify UserProvider.DeleteSecrets doc with all-or-nothing semantics
- Add mixed-input test case for UserProvider.DeleteSecrets confirming
  inner provider is not called when any key in the list is reserved

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@amirejaz amirejaz requested a review from aponcedeleonch March 30, 2026 12:52
@github-actions github-actions bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Mar 30, 2026
Copy link
Copy Markdown
Member

@aponcedeleonch aponcedeleonch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

@amirejaz amirejaz merged commit c904ade into main Mar 30, 2026
70 of 71 checks passed
@amirejaz amirejaz deleted the scoped-secret-providers branch March 30, 2026 13:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/XL Extra large PR: 1000+ lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scoped secret store: core providers and system key isolation

3 participants