fix(security): audit web search keys for all bundled providers#56540
fix(security): audit web search keys for all bundled providers#56540hydro13 merged 1 commit intoopenclaw:mainfrom
Conversation
Greptile SummaryThis PR fixes a security audit gap where Key points:
Confidence Score: 5/5Safe to merge — all remaining findings are non-blocking P2 suggestions around test coverage and operator semantics. No P0 or P1 issues found. The new implementation is logically sound and all changed code paths have test coverage for the new providers. The two P2 notes (missing Brave/Perplexity regression tests and the ?? operator behaviour) are improvement suggestions that do not affect correctness of the shipped fix. No files require special attention, though adding Brave and Perplexity cases to the new test suite would improve regression safety.
|
| Filename | Overview |
|---|---|
| src/security/audit-extra.sync.ts | Replaces hardcoded Brave/Perplexity credential checks in hasWebSearchKey with a provider-aware loop over resolveBundledPluginWebSearchProviders. Logic is correct; the ?? fallback to getCredentialValue(search) intentionally preserves legacy-path coverage for migrated providers. |
| src/security/audit-extra.sync.test.ts | Adds six new regression cases for the newly supported providers (Gemini, Grok, Kimi, Moonshot, OpenRouter). Does not include regression tests for the two previously hardcoded providers (Brave, Perplexity) whose paths are now exercised through the generic mechanism. |
Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/security/audit-extra.sync.ts
Line: 343-344
Comment:
**`??` falls through on both absent and `null`/`undefined` return values**
The current expression:
```ts
provider.getConfiguredCredentialValue?.(cfg) ?? provider.getCredentialValue(search)
```
will fall through to `getCredentialValue(search)` not only when `getConfiguredCredentialValue` is absent (optional chaining returns `undefined`), but also when it is **defined** and explicitly returns `null` or `undefined` (e.g., "no credential at this config path"). This means the legacy `cfg.tools.web.search` path is always consulted as a fallback even for providers that have fully migrated to `getConfiguredCredentialValue`.
If the intent is backward-compat double-checking, that is fine and working as designed. However, if the intent is "use only the new path once the method is defined", the check should use a ternary instead:
```ts
const configuredCredential = provider.getConfiguredCredentialValue
? provider.getConfiguredCredentialValue(cfg)
: provider.getCredentialValue(search);
```
A comment clarifying the intended dual-path behaviour would prevent future confusion regardless.
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: src/security/audit-extra.sync.test.ts
Line: 59-107
Comment:
**Missing regression tests for Brave and Perplexity**
The PR replaces the previously hardcoded checks for `BRAVE_API_KEY` and `PERPLEXITY_API_KEY` with the generic `resolveBundledPluginWebSearchProviders` mechanism, but no regression test cases cover these two providers in the new `collectSmallModelRiskFindings` suite.
If the Brave or Perplexity provider's `envVars` array is ever accidentally modified (or the provider's plugin ID changes), the regression would be silent — neither the old hardcoded path nor any new test would catch it. Consider adding equivalent `it.each` entries for `BRAVE_API_KEY` and `PERPLEXITY_API_KEY` to match the pattern used for the six new providers.
How can I resolve this? If you propose a fix, please make it concise.Reviews (1): Last reviewed commit: "fix(security): audit web search keys for..." | Re-trigger Greptile
| const configuredCredential = | ||
| provider.getConfiguredCredentialValue?.(cfg) ?? provider.getCredentialValue(search); |
There was a problem hiding this comment.
?? falls through on both absent and null/undefined return values
The current expression:
provider.getConfiguredCredentialValue?.(cfg) ?? provider.getCredentialValue(search)will fall through to getCredentialValue(search) not only when getConfiguredCredentialValue is absent (optional chaining returns undefined), but also when it is defined and explicitly returns null or undefined (e.g., "no credential at this config path"). This means the legacy cfg.tools.web.search path is always consulted as a fallback even for providers that have fully migrated to getConfiguredCredentialValue.
If the intent is backward-compat double-checking, that is fine and working as designed. However, if the intent is "use only the new path once the method is defined", the check should use a ternary instead:
const configuredCredential = provider.getConfiguredCredentialValue
? provider.getConfiguredCredentialValue(cfg)
: provider.getCredentialValue(search);A comment clarifying the intended dual-path behaviour would prevent future confusion regardless.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/security/audit-extra.sync.ts
Line: 343-344
Comment:
**`??` falls through on both absent and `null`/`undefined` return values**
The current expression:
```ts
provider.getConfiguredCredentialValue?.(cfg) ?? provider.getCredentialValue(search)
```
will fall through to `getCredentialValue(search)` not only when `getConfiguredCredentialValue` is absent (optional chaining returns `undefined`), but also when it is **defined** and explicitly returns `null` or `undefined` (e.g., "no credential at this config path"). This means the legacy `cfg.tools.web.search` path is always consulted as a fallback even for providers that have fully migrated to `getConfiguredCredentialValue`.
If the intent is backward-compat double-checking, that is fine and working as designed. However, if the intent is "use only the new path once the method is defined", the check should use a ternary instead:
```ts
const configuredCredential = provider.getConfiguredCredentialValue
? provider.getConfiguredCredentialValue(cfg)
: provider.getCredentialValue(search);
```
A comment clarifying the intended dual-path behaviour would prevent future confusion regardless.
How can I resolve this? If you propose a fix, please make it concise.| it.each([ | ||
| { | ||
| name: "gemini plugin config", | ||
| cfg: { | ||
| ...baseCfg, | ||
| plugins: { | ||
| entries: { | ||
| google: { enabled: true, config: { webSearch: { apiKey: "AIza-test" } } }, | ||
| }, | ||
| }, | ||
| } satisfies OpenClawConfig, | ||
| env: {}, | ||
| }, | ||
| { | ||
| name: "gemini env key", | ||
| cfg: baseCfg, | ||
| env: { GEMINI_API_KEY: "AIza-test" }, | ||
| }, | ||
| { | ||
| name: "grok env key", | ||
| cfg: baseCfg, | ||
| env: { XAI_API_KEY: "xai-test" }, | ||
| }, | ||
| { | ||
| name: "kimi env key", | ||
| cfg: baseCfg, | ||
| env: { KIMI_API_KEY: "sk-kimi-test" }, | ||
| }, | ||
| { | ||
| name: "moonshot env key", | ||
| cfg: baseCfg, | ||
| env: { MOONSHOT_API_KEY: "sk-moonshot-test" }, | ||
| }, | ||
| { | ||
| name: "openrouter env fallback", | ||
| cfg: baseCfg, | ||
| env: { OPENROUTER_API_KEY: "sk-or-v1-test" }, | ||
| }, | ||
| ])("$name", ({ cfg, env }) => { | ||
| const [finding] = collectSmallModelRiskFindings({ | ||
| cfg, | ||
| env, | ||
| }); | ||
|
|
||
| expect(finding?.checkId).toBe("models.small_params"); | ||
| expect(finding?.severity).toBe("critical"); | ||
| expect(finding?.detail).toContain("web_search"); | ||
| }); | ||
| }); |
There was a problem hiding this comment.
Missing regression tests for Brave and Perplexity
The PR replaces the previously hardcoded checks for BRAVE_API_KEY and PERPLEXITY_API_KEY with the generic resolveBundledPluginWebSearchProviders mechanism, but no regression test cases cover these two providers in the new collectSmallModelRiskFindings suite.
If the Brave or Perplexity provider's envVars array is ever accidentally modified (or the provider's plugin ID changes), the regression would be silent — neither the old hardcoded path nor any new test would catch it. Consider adding equivalent it.each entries for BRAVE_API_KEY and PERPLEXITY_API_KEY to match the pattern used for the six new providers.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/security/audit-extra.sync.test.ts
Line: 59-107
Comment:
**Missing regression tests for Brave and Perplexity**
The PR replaces the previously hardcoded checks for `BRAVE_API_KEY` and `PERPLEXITY_API_KEY` with the generic `resolveBundledPluginWebSearchProviders` mechanism, but no regression test cases cover these two providers in the new `collectSmallModelRiskFindings` suite.
If the Brave or Perplexity provider's `envVars` array is ever accidentally modified (or the provider's plugin ID changes), the regression would be silent — neither the old hardcoded path nor any new test would catch it. Consider adding equivalent `it.each` entries for `BRAVE_API_KEY` and `PERPLEXITY_API_KEY` to match the pattern used for the six new providers.
How can I resolve this? If you propose a fix, please make it concise.There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 70d016675a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if (typeof value === "string") { | ||
| return value.trim().length > 0; | ||
| } | ||
| return value !== undefined && value !== null; |
There was a problem hiding this comment.
Validate credential shape before treating it as configured
hasConfiguredCredentialValue currently treats any non-null/undefined non-string as a valid credential. That makes values like false, 0, or unresolved secret-ref objects count as “configured,” so hasWebSearchKey can report web_search exposure even when no usable API key exists at runtime. This creates false critical models.small_params findings for malformed or unresolved credentials; the check should mirror runtime credential normalization (non-empty string or resolvable secret input) instead of accepting arbitrary objects/primitives.
Useful? React with 👍 / 👎.
hasWebSearchKey() was hardcoded to only check Brave and Perplexity credentials. Replace with provider-aware check using resolveBundledPluginWebSearchProviders() so Gemini, Grok/XAI, Kimi, Moonshot, and OpenRouter credentials are recognized by the audit. Add focused regression tests for each provider. Fixes openclaw#34509
70d0166 to
07b57da
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 07b57daaa6
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| return Boolean( | ||
| search?.apiKey || search?.perplexity?.apiKey || env.BRAVE_API_KEY || env.PERPLEXITY_API_KEY, | ||
| ); | ||
| return hasBundledWebSearchCredential({ config: cfg, env }); |
There was a problem hiding this comment.
Cache web-search key detection outside the model loop
hasWebSearchKey now calls hasBundledWebSearchCredential, which resolves bundled providers through plugin-resolution paths instead of a constant-time field/env check. collectSmallModelRiskFindings invokes this for every small model, so a config with multiple small-model entries repeatedly re-runs bundled provider resolution work (including manifest/runtime registry traversal), turning this audit path into avoidable synchronous overhead; computing this once per audit run (or memoizing in hasWebSearchKey) would keep the check cheap and prevent latency spikes in larger configs.
Useful? React with 👍 / 👎.
Add missing changelog entries for PRs openclaw#56500, openclaw#56540, openclaw#56555, openclaw#56567, openclaw#56573, openclaw#56587, openclaw#56595, openclaw#56612, openclaw#56620.
…law#56540) hasWebSearchKey() was hardcoded to only check Brave and Perplexity credentials. Replace with provider-aware check using resolveBundledPluginWebSearchProviders() so Gemini, Grok/XAI, Kimi, Moonshot, and OpenRouter credentials are recognized by the audit. Add focused regression tests for each provider. Fixes openclaw#34509
Add missing changelog entries for PRs openclaw#56500, openclaw#56540, openclaw#56555, openclaw#56567, openclaw#56573, openclaw#56587, openclaw#56595, openclaw#56612, openclaw#56620.
…law#56540) hasWebSearchKey() was hardcoded to only check Brave and Perplexity credentials. Replace with provider-aware check using resolveBundledPluginWebSearchProviders() so Gemini, Grok/XAI, Kimi, Moonshot, and OpenRouter credentials are recognized by the audit. Add focused regression tests for each provider. Fixes openclaw#34509
Add missing changelog entries for PRs openclaw#56500, openclaw#56540, openclaw#56555, openclaw#56567, openclaw#56573, openclaw#56587, openclaw#56595, openclaw#56612, openclaw#56620.
…law#56540) hasWebSearchKey() was hardcoded to only check Brave and Perplexity credentials. Replace with provider-aware check using resolveBundledPluginWebSearchProviders() so Gemini, Grok/XAI, Kimi, Moonshot, and OpenRouter credentials are recognized by the audit. Add focused regression tests for each provider. Fixes openclaw#34509
Add missing changelog entries for PRs openclaw#56500, openclaw#56540, openclaw#56555, openclaw#56567, openclaw#56573, openclaw#56587, openclaw#56595, openclaw#56612, openclaw#56620.
…law#56540) hasWebSearchKey() was hardcoded to only check Brave and Perplexity credentials. Replace with provider-aware check using resolveBundledPluginWebSearchProviders() so Gemini, Grok/XAI, Kimi, Moonshot, and OpenRouter credentials are recognized by the audit. Add focused regression tests for each provider. Fixes openclaw#34509
Add missing changelog entries for PRs openclaw#56500, openclaw#56540, openclaw#56555, openclaw#56567, openclaw#56573, openclaw#56587, openclaw#56595, openclaw#56612, openclaw#56620.
…law#56540) hasWebSearchKey() was hardcoded to only check Brave and Perplexity credentials. Replace with provider-aware check using resolveBundledPluginWebSearchProviders() so Gemini, Grok/XAI, Kimi, Moonshot, and OpenRouter credentials are recognized by the audit. Add focused regression tests for each provider. Fixes openclaw#34509
Add missing changelog entries for PRs openclaw#56500, openclaw#56540, openclaw#56555, openclaw#56567, openclaw#56573, openclaw#56587, openclaw#56595, openclaw#56612, openclaw#56620.
…law#56540) hasWebSearchKey() was hardcoded to only check Brave and Perplexity credentials. Replace with provider-aware check using resolveBundledPluginWebSearchProviders() so Gemini, Grok/XAI, Kimi, Moonshot, and OpenRouter credentials are recognized by the audit. Add focused regression tests for each provider. Fixes openclaw#34509
Add missing changelog entries for PRs openclaw#56500, openclaw#56540, openclaw#56555, openclaw#56567, openclaw#56573, openclaw#56587, openclaw#56595, openclaw#56612, openclaw#56620.
…law#56540) hasWebSearchKey() was hardcoded to only check Brave and Perplexity credentials. Replace with provider-aware check using resolveBundledPluginWebSearchProviders() so Gemini, Grok/XAI, Kimi, Moonshot, and OpenRouter credentials are recognized by the audit. Add focused regression tests for each provider. Fixes openclaw#34509
Add missing changelog entries for PRs openclaw#56500, openclaw#56540, openclaw#56555, openclaw#56567, openclaw#56573, openclaw#56587, openclaw#56595, openclaw#56612, openclaw#56620.
Summary
hasWebSearchKey()was hardcoded to only check Brave and Perplexity credentialsresolveBundledPluginWebSearchProviders()Root Cause
The audit function
hasWebSearchKey()insrc/security/audit-extra.sync.tswas still hardcoded to legacy surfaces (Brave config/env + Perplexity config/env). Web search has moved to provider-owned metadata and plugin config paths, so newer providers were invisible to the audit.Change Type
Testing
All 15 tests pass including 6 new regression tests:
GEMINI_API_KEY)XAI_API_KEY)KIMI_API_KEY)MOONSHOT_API_KEY)OPENROUTER_API_KEY)Fixes #34509