Skip to content

Feature: Secure Secrets Management Tool — API Key Ingestion, Scoped Access, Redaction, and Skill Requirements #410

@teknium1

Description

@teknium1

Overview

Hermes Agent needs a first-class secrets management tool that lets the agent safely ingest, store, use, and protect API keys and other credentials. Today, secrets are stored as plaintext in ~/.hermes/.env, every subprocess gets the full environment, file tools expose raw secrets, and there is no mechanism for skills to declare "I need a Twilio API key" and have the agent securely prompt the user for it.

This is the umbrella issue for end-to-end secure secret management. It subsumes and coordinates with:

The core challenge: Skills like Twilio (#409) require API keys. The agent needs to know it's missing a key, prompt the user to provide it, store it securely, and use it in CLI commands — all without the key ever appearing in chat logs, terminal output, session history, or LLM context.


Research Findings

How Other Agents Handle Secrets (State of the Art)

Most agents have NO built-in secret protection. This is a known unsolved problem:

Agent Approach Issues
Claude Code Reads .env silently; .claudeignore unreliable; deny rules in settings.json Known incidents: pushed API keys to GitHub (#20966 in their repo)
Cursor File indexing sees .env files; no vault Attempted to upload files containing hidden API keys to cloud
Aider .env files, CLI args, YAML config Docs suggest putting keys on command line (shell history exposure)
Cline VS Code context.secrets API (encrypted) Best of the IDE agents, but still exposes in file operations
Codex "Environment variables" + "Secrets" (setup-phase only) Secrets only available during container setup, not runtime
Devin Full system access, no secret isolation Pillar Security: "Insecure Credential Handling" is critical risk

Emerging solutions:

  • AgentSecrets — Zero-knowledge proxy. Agent NEVER sees key values; proxy injects keys at the transport layer. Agent only sees API responses. Uses OS keychain (macOS Keychain, Linux keyring). Supports MCP integration.

  • ExitBox — AES-256 encrypted vault with Argon2id key derivation. Every secret read/write requires y/n approval via tmux. .env files mounted as /dev/null inside containers to force vault usage. AGPL-3.0 (concepts only).

  • PII-Shield — Go sidecar that intercepts stdout. Uses Shannon entropy + bigram analysis (not regex) to detect secrets. Deterministic HMAC redaction: [HIDDEN:8f2a1b] preserves referential integrity for debugging.

  • Codenotary's Layered Approach — 3 layers: (1) Access control - block opening secret files, (2) Context cleaning - scrub data before LLM sees it, (3) Output filtering - mask patterns in output.

Key insight from research: "If an agent can see it, it can leak it." The safest approach is preventing secrets from entering the agent's context at all, using proxy-based or placeholder-based architectures.

Current Hermes Secret Handling

How .env is loaded (cli.py lines 52-68):

  • python-dotenv loads ~/.hermes/.env at process startup
  • Values injected into os.environ
  • hermes_cli/config.py has save_env_value(key, value) for writing to .env

Where redaction IS applied:

  • Terminal tool output (tools/terminal_tool.py line 1041): redact_sensitive_text() on all command output
  • Log files (run_agent.py line 257): RedactingFormatter wraps all log formatters
  • Gateway logs (gateway/run.py): Both gateway.log and errors.log use RedactingFormatter

Where redaction is NOT applied (#363):

  • read_file tool — returns raw file content including secrets
  • search_files tool — returns raw match content
  • patch tool — shows raw content in diffs
  • execute_code tool — script output not redacted

The agent/redact.py module (116 lines):

  • Matches known prefixes: sk-*, ghp_*, xox*-*, AIza*, pplx-*, fal_*, fc-*, bb_live_*
  • Matches ENV assignment patterns: API_KEY=value, TOKEN=value, SECRET=value
  • Matches JSON fields: "apiKey": "value", "token": "value"
  • Missing patterns: Twilio SIDs/tokens, Stripe keys, AWS AKIA, JWTs, private key blocks, high-entropy strings

Critical gaps identified in the codebase:

  1. Terminal subprocess env leakagetools/environments/local.py passes ALL os.environ to subprocesses. The agent's commands have full access to every secret via $VAR expansion.
  2. Inconsistent env filteringtools/mcp_tool.py carefully filters env vars (only passes _SAFE_ENV_KEYS: PATH, HOME, USER, LANG, etc.). Terminal tool passes everything. Inconsistent.
  3. No encryption at rest~/.hermes/.env is plaintext readable by anyone with file access.
  4. No per-skill scoping — All secrets available to all tools equally.
  5. No secure ingestion flow — No way for agent to prompt user for a key without it appearing in chat history.
  6. Write protection is incompletefile_operations.py blocks write_file to .env, but agent can still terminal("echo KEY=val >> ~/.hermes/.env").

Proposed Solution: secrets Tool

A new first-class tool that provides the complete lifecycle: declare → prompt → store → use → protect.

Tool API

Tool: secrets

Actions:

1. list — Show what secrets exist (names only, never values)
   secrets(action="list")
   Returns: {"secrets": ["TWILIO_ACCOUNT_SID", "TWILIO_AUTH_TOKEN", ...],
             "missing_for_skills": {"twilio": ["TWILIO_AUTH_TOKEN"]}}

2. check — Verify if specific secrets are configured
   secrets(action="check", keys=["TWILIO_ACCOUNT_SID", "TWILIO_AUTH_TOKEN"])
   Returns: {"configured": ["TWILIO_ACCOUNT_SID"],
             "missing": ["TWILIO_AUTH_TOKEN"]}

3. request — Ask the user to provide a secret (secure input)
   secrets(action="request",
           key="TWILIO_AUTH_TOKEN",
           description="Twilio Auth Token for making phone calls",
           instructions="Find this at https://console.twilio.com/ under Account > API Keys")
   - CLI: prompts with getpass() — input hidden, never in readline history
   - Telegram/Discord: sends DM with instructions, accepts reply,
     immediately deletes the message containing the key
   - The VALUE never enters the agent's context or conversation history
   Returns: {"stored": true, "key": "TWILIO_AUTH_TOKEN"}

4. delete — Remove a secret
   secrets(action="delete", key="TWILIO_AUTH_TOKEN")

5. inject — Make specific secrets available for the next terminal command
   secrets(action="inject", keys=["TWILIO_ACCOUNT_SID", "TWILIO_AUTH_TOKEN"])
   Returns: {"injected": true, "keys": ["TWILIO_ACCOUNT_SID", "TWILIO_AUTH_TOKEN"]}
   (Sets up scoped env for the next terminal() call only)

Secure Ingestion Flow

The critical UX challenge: how does a user enter an API key without it appearing in chat?

CLI Mode:

Agent: The Twilio skill requires TWILIO_AUTH_TOKEN.
       Find it at https://console.twilio.com/

       Please enter your Twilio Auth Token:
> [input is hidden via getpass(), not echoed, not logged]

Agent: ✓ TWILIO_AUTH_TOKEN has been securely stored.

Implementation: The secrets(action="request") call triggers a platform-specific secure input handler:

  • CLI: Uses Python getpass.getpass() — input is not echoed, not in readline history
  • Telegram: Sends a DM to the user asking for the key. When the user replies, the bot immediately deletes the message containing the key, stores it, and confirms. The key never enters the conversation history that feeds the LLM.
  • Discord: Same pattern — DM, store, delete the message
  • WhatsApp: Sends instructions, accepts reply (deletion limited on WhatsApp)

The key NEVER enters the LLM conversation context. The tool handles the entire flow internally and only returns {"stored": true} to the agent.

Secure Usage: Environment Variable Scoping

Today, terminal_tool.py passes the entire os.environ to subprocesses. This must change.

Proposed: Tiered environment model

Tier 1: SAFE — always available to subprocesses
  PATH, HOME, USER, LANG, TERM, SHELL, TMPDIR, LC_ALL, EDITOR, XDG_*

Tier 2: TOOL-MANAGED — available only when the tool explicitly
  requests them (e.g., web_tools needs FIRECRAWL_API_KEY)
  Tool registration declares which env vars it needs via injects_env

Tier 3: USER-REQUESTED — available only when agent explicitly
  uses secrets(action="inject") before a terminal command
  Agent says "I need TWILIO_ACCOUNT_SID for this curl call"
  Terminal tool injects only those vars for that command

For the terminal tool specifically, a new parameter:

terminal(
    command="curl -X POST https://api.twilio.com/... -u $TWILIO_ACCOUNT_SID:$TWILIO_AUTH_TOKEN",
    env_keys=["TWILIO_ACCOUNT_SID", "TWILIO_AUTH_TOKEN"]
)

The terminal tool would:

  1. Start with only SAFE_ENV vars
  2. Add only the explicitly listed env_keys from the secret store
  3. The command can reference $TWILIO_ACCOUNT_SID normally — shell expansion works
  4. The output is redacted via redact_sensitive_text() as usual
  5. The actual values never appear in the tool call arguments or response

This follows the same pattern already established by tools/mcp_tool.py which uses _SAFE_ENV_KEYS — we're extending it to the terminal tool.

Skill Secret Requirements

Skills should be able to declare what secrets they need in SKILL.md frontmatter:

---
name: twilio
requires_secrets:
  - key: TWILIO_ACCOUNT_SID
    description: "Twilio Account SID"
    instructions: "Find at https://console.twilio.com/"
  - key: TWILIO_AUTH_TOKEN
    description: "Twilio Auth Token"
    instructions: "Find at https://console.twilio.com/"
  - key: TWILIO_PHONE_NUMBER
    description: "Your Twilio phone number (E.164 format)"
    instructions: "Buy a number at https://console.twilio.com/phone-numbers"
---

When the agent loads a skill with requires_secrets, it:

  1. Checks if all required secrets are configured via secrets(action="check")
  2. If any are missing, uses secrets(action="request") for each one
  3. Only proceeds with the skill instructions once all secrets are available

Storage Security

Phase 1 (simple): Continue using ~/.hermes/.env but with better protection:

  • chmod 600 ~/.hermes/.env (owner-only read/write)
  • Block ALL tools from reading .env content (extend blocklist to terminal too)
  • Only the secrets tool can access .env values

Phase 2 (encrypted): Migrate to encrypted storage:

  • AES-256 encrypted file at ~/.hermes/secrets.enc
  • Key derived from machine-specific entropy via Argon2id
  • Optional master password for additional security
  • Python cryptography package

Phase 3 (OS keychain): Use platform-native secure storage:

  • macOS: Keychain via keyring package
  • Linux: Secret Service (GNOME Keyring / KWallet) via keyring
  • Fallback: encrypted file from Phase 2

Implementation Plan

Tool Classification

This is a tool (not a skill) because:

  • Requires custom Python integration for secure input handling (getpass, platform-specific message deletion)
  • Needs deterministic processing logic (encryption, env filtering) that can't be "best effort" LLM interpretation
  • Must integrate deeply with the terminal tool's subprocess environment
  • Handles sensitive data that requires precise control over what enters/exits the LLM context

Phased Rollout

Phase 1: Core Secrets Tool + Redaction Fix (MVP)

  • New tools/secrets_tool.py with list, check, request, delete, inject actions
  • Secure input: getpass() for CLI, DM+delete for gateway platforms
  • Values stored in ~/.hermes/.env (existing infrastructure)
  • File permissions: chmod 600 on .env
  • Block all tools from reading .env content (extend file_operations blocklist)
  • Fix Security: File Tool Output Redaction Gap — Secrets Exposed via read_file but Masked via Terminal #363: Apply redact_sensitive_text() to file tool outputs
  • Expand redaction patterns: add Twilio SIDs, AWS AKIA, Stripe keys, JWTs, high-entropy strings
  • requires_secrets field in SKILL.md frontmatter parsing
  • Deliverables: tools/secrets_tool.py + enhanced agent/redact.py + skill frontmatter support

Phase 2: Environment Scoping + Terminal Integration

  • Tiered env model: SAFE_ENV + TOOL-MANAGED + USER-REQUESTED
  • env_keys parameter on terminal tool for explicit secret injection
  • Tool registry injects_env field for automatic per-tool env scoping
  • Default transition: terminal subprocess gets only SAFE_ENV (with opt-in migration period)
  • Shannon entropy-based secret detection in output (inspired by PII-Shield)
  • Deliverables: Enhanced terminal_tool.py + env scoping infrastructure

Phase 3: Encrypted Storage + Skill Onboarding

  • Encrypted secret store with Argon2id + AES-256
  • Migration tool from plaintext .env to encrypted store
  • OS keychain support via keyring package
  • Skill onboarding wizard: auto-prompt for missing keys when loading skills
  • Secret rotation reminders (track creation date, warn after configurable days)
  • Per-secret access scoping (which tools/skills can access which secrets)
  • Deliverables: Encrypted storage + keyring integration + onboarding flow

Phase 4: Advanced Security

  • Audit log: track every secret access (which tool, when, for what command)
  • Ephemeral secrets: auto-expiring keys for temporary access
  • Integration with external vaults (HashiCorp Vault, AWS Secrets Manager)
  • Network-level protection: coordinate with Address outbound threats #129 to restrict outbound access per-secret
  • Deliverables: Audit system + vault integrations

Pros & Cons

Pros

Cons / Risks

  • Complexity — Environment scoping (Phase 2) changes terminal tool behavior. Needs careful backward compatibility.
  • False positives in redaction — Entropy-based detection may mask legitimate base64, UUIDs, or long hashes.
  • Agent confusion — If the agent sees [REDACTED] in outputs, it may try to work around it.
  • Breaking change — Phase 2 env scoping changes the default from "pass everything" to "pass only safe vars."
  • DM-based ingestion — Not all platforms support message deletion. WhatsApp doesn't allow bots to delete user messages.
  • Encryption key management — Machine-derived keys may change. Need recovery mechanism.

Open Questions

  1. Backward compatibility for env scoping — Should Phase 2 be opt-in or opt-out? Opt-in is safer but means existing commands still leak secrets.
  2. Should the agent EVER see secret values? — Current design: never. But some use cases require it (agent writing a config file). Feature: Agent-Vault Skill — Placeholder-Based Secret Management for Config Files #364's placeholder approach handles this.
  3. Gateway message deletion reliability — Telegram has deleteMessage API. Discord has delete(). WhatsApp has limited deletion support. What's the fallback?
  4. Migration path — How to migrate existing users from plaintext .env to encrypted storage without disruption?
  5. Relationship to Feature: Agent-Vault Skill — Placeholder-Based Secret Management for Config Files #364 — Should this tool REPLACE agent-vault or work alongside it? They're complementary but need clear boundaries.
  6. MCP server secrets — Should the secrets tool manage MCP server credentials too?
  7. hermes config set integration — Today hermes config set TWILIO_AUTH_TOKEN=xxx writes to .env. Should this command route through the secrets tool instead?

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions