Skip to content

fix: normalizeProviders flip-flop bug when using env var apiKey references#47305

Closed
slideshow-dingo wants to merge 7 commits intoopenclaw:mainfrom
slideshow-dingo:fix/normalizeProviders-flipflop-bug
Closed

fix: normalizeProviders flip-flop bug when using env var apiKey references#47305
slideshow-dingo wants to merge 7 commits intoopenclaw:mainfrom
slideshow-dingo:fix/normalizeProviders-flipflop-bug

Conversation

@slideshow-dingo
Copy link
Copy Markdown

Bug Description

When openclaw.json configures a provider apiKey using an env var reference, the normalizeProviders() function creates a flip-flop cycle:

  1. First normalization: Writes the env var name to models.json
  2. User manually fixes: Changes to the resolved value
  3. Next normalization: Converts it back to the env var name

This causes models.json to alternate between env var name and resolved value on every normalization.

Root Cause

Located in src/agents/models-config.providers.ts lines 504-519 (v2026.3.13) - a reverse-lookup block that converts resolved values back to env var names.

Fix

Remove the reverse-lookup block entirely. models.json should contain resolved values, not env var names.

Testing

Added src/agents/models-config.providers.flipflop.test.ts with:

  • Bug reproduction test (passes before fix, fails after)
  • Fix verification tests (pass after fix)
  • Regression test for {source: "env"} config

Related Issues

OpenClaw Version: v2026.3.13

…ences

Removes reverse-lookup logic that converts resolved apiKey values back to
env var names, causing models.json to flip-flop on successive normalizations.

Bug: When openclaw.json configures apiKey using { source: 'env' }, the
resolved value is written to models.json. On next normalization, the code
detects the resolved value matches an env var and converts it back to the
env var name, causing instability.

Fix: Delete lines 504-519 in models-config.providers.ts that perform the
reverse-lookup conversion. models.json should contain resolved values, not
env var names.

Test: Add models-config.providers.flipflop.test.ts demonstrating the bug
and verifying the fix preserves resolved values and manual edits.

Related: #14808 (security: plaintext in cache), #11202 (keys in prompt)
@openclaw-barnacle openclaw-barnacle Bot added agents Agent runtime and tooling size: M labels Mar 15, 2026
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Mar 15, 2026

Greptile Summary

This PR fixes a flip-flop bug in normalizeProviders() where providers configured with a plain-string apiKey that happened to equal an env-var value would have that key silently converted to the env-var name on every normalization run, causing models.json to revert user edits. The fix makes the reverse-lookup conditional on the original source config (sourceProviders) actually containing an { source: "env" } SecretRef for that provider, so the conversion only fires when it was genuinely configured as an env-var reference.

Key observations:

  • The sole production caller (models-config.plan.ts) already passes sourceProviders via resolved.sourceConfigForSecrets, so the security invariant from issue [Bug]: v2026.3.2: custom provider apiKey from env/SecretRef is persisted as plaintext into state agents/*/agent/models.json #38757 (preventing plaintext persistence when env refs are in use) is preserved.
  • All four previously flagged thread issues (missing BUG REPRODUCTION test, ReferenceError on normalized1, wrong secretref-env: marker prefix, leftover double blank line) are resolved in this HEAD commit.
  • The ENV_VAR_NAME_RE constant move and the abort.test.ts mock rename are clean, unrelated housekeeping.
  • One subtle ordering concern exists: after the new reverse-lookup sets normalizedProvider.apiKey, execution falls through to normalizeApiKeyConfig(configuredApiKey), which re-reads the original value. A future change to upstream SecretRef detection could make both branches fire for the same input, silently overwriting the reverse-lookup result. Adding a guard (skip normalizeApiKeyConfig when normalizedProvider.apiKey has already been updated) would make the mutual exclusivity explicit.
  • The new "manual edits preserved" test constructs its edited input from originalProviders rather than from the output of the first normalization call, which means a round-trip regression (first pass mutates, second pass reverts) would not be caught.

Confidence Score: 4/5

  • Safe to merge; the fix is logically correct and the security path for env-var references is preserved by the production caller.
  • The core logic change is sound and correctly scoped. The production caller already passes sourceProviders, so the [Bug]: v2026.3.2: custom provider apiKey from env/SecretRef is persisted as plaintext into state agents/*/agent/models.json #38757 security fix remains effective. The only concerns are a subtle fall-through ordering assumption in models-config.providers.ts (benign today but fragile under future refactoring) and a test that doesn't fully exercise the chained-normalization scenario it describes — neither blocks merging.
  • Mild attention to src/agents/models-config.providers.ts lines 528–536 (fall-through interaction between the new reverse-lookup block and normalizeApiKeyConfig) and src/agents/models-config.providers.flipflop.test.ts lines 112–118 (non-chained normalization test).

Comments Outside Diff (1)

  1. src/agents/models-config.providers.ts, line 528-536 (link)

    Reverse-lookup result could be overwritten by the fall-through normalizeApiKeyConfig path

    After the conditional reverse-lookup (lines 513–526) sets normalizedProvider.apiKey to the env-var name, execution falls through to normalizeApiKeyConfig(configuredApiKey), which still operates on the original configuredApiKey. If that call returns a value different from configuredApiKey (e.g. stripping ${...} braces), the spread on lines 532–535 would silently overwrite the env-var name the reverse-lookup just produced.

    In practice this is benign today — any configuredApiKey that would be transformed by normalizeApiKeyConfig (i.e. matching ${VAR}) would have produced a non-null configuredApiKeyRef upstream, routing it through the if branch before this else if is ever reached. But the correctness depends on that upstream invariant remaining intact, and the fall-through ordering makes the assumption easy to miss in future refactors.

    A simple guard (checking normalizedProvider.apiKey === configuredApiKey before calling normalizeApiKeyConfig) would make the mutual exclusivity explicit and prevent any future breakage if the ${VAR} detection logic upstream ever changes.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: src/agents/models-config.providers.ts
    Line: 528-536
    
    Comment:
    **Reverse-lookup result could be overwritten by the fall-through `normalizeApiKeyConfig` path**
    
    After the conditional reverse-lookup (lines 513–526) sets `normalizedProvider.apiKey` to the env-var name, execution falls through to `normalizeApiKeyConfig(configuredApiKey)`, which still operates on the **original** `configuredApiKey`. If that call returns a value different from `configuredApiKey` (e.g. stripping `${...}` braces), the spread on lines 532–535 would silently overwrite the env-var name the reverse-lookup just produced.
    
    In practice this is benign today — any `configuredApiKey` that would be transformed by `normalizeApiKeyConfig` (i.e. matching `${VAR}`) would have produced a non-null `configuredApiKeyRef` upstream, routing it through the `if` branch before this `else if` is ever reached. But the correctness depends on that upstream invariant remaining intact, and the fall-through ordering makes the assumption easy to miss in future refactors.
    
    A simple guard (checking `normalizedProvider.apiKey === configuredApiKey` before calling `normalizeApiKeyConfig`) would make the mutual exclusivity explicit and prevent any future breakage if the `${VAR}` detection logic upstream ever changes.
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/agents/models-config.providers.ts
Line: 528-536

Comment:
**Reverse-lookup result could be overwritten by the fall-through `normalizeApiKeyConfig` path**

After the conditional reverse-lookup (lines 513–526) sets `normalizedProvider.apiKey` to the env-var name, execution falls through to `normalizeApiKeyConfig(configuredApiKey)`, which still operates on the **original** `configuredApiKey`. If that call returns a value different from `configuredApiKey` (e.g. stripping `${...}` braces), the spread on lines 532–535 would silently overwrite the env-var name the reverse-lookup just produced.

In practice this is benign today — any `configuredApiKey` that would be transformed by `normalizeApiKeyConfig` (i.e. matching `${VAR}`) would have produced a non-null `configuredApiKeyRef` upstream, routing it through the `if` branch before this `else if` is ever reached. But the correctness depends on that upstream invariant remaining intact, and the fall-through ordering makes the assumption easy to miss in future refactors.

A simple guard (checking `normalizedProvider.apiKey === configuredApiKey` before calling `normalizeApiKeyConfig`) would make the mutual exclusivity explicit and prevent any future breakage if the `${VAR}` detection logic upstream ever changes.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/agents/models-config.providers.flipflop.test.ts
Line: 112-118

Comment:
**"Manual edits preserved" test does not chain the two normalizations**

The first `normalizeProviders` call result is discarded (not assigned). The `editedProviders` object is then constructed from `originalProviders` directly and normalized in isolation. This correctly exercises the no-`sourceProviders` code path, but it does not cover the scenario described by the comment — where the *output* of a first normalization is manually edited and then re-normalized. A regression where the first pass mutates the value in a way that causes the second pass to revert it would go undetected.

For stronger coverage, consider passing the output of the first call as the base for the edit, so the test genuinely validates round-trip stability.

How can I resolve this? If you propose a fix, please make it concise.

Last reviewed commit: 111020b

Comment thread src/agents/models-config.providers.flipflop.test.ts Outdated
Comment thread src/agents/models-config.providers.flipflop.test.ts Outdated
Comment thread src/agents/models-config.providers.ts Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fd29a9cbde

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/agents/models-config.providers.ts
Comment thread src/agents/models-config.providers.flipflop.test.ts Outdated
@openclaw-barnacle openclaw-barnacle Bot added the channel: whatsapp-web Channel integration: whatsapp-web label Mar 15, 2026
@slideshow-dingo slideshow-dingo marked this pull request as ready for review March 15, 2026 22:51
Comment thread src/agents/models-config.providers.flipflop.test.ts Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 05db60fe60

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/agents/models-config.providers.flipflop.test.ts Outdated
Comment thread src/agents/models-config.providers.flipflop.test.ts Outdated
@slideshow-dingo slideshow-dingo marked this pull request as draft March 15, 2026 22:57
@slideshow-dingo slideshow-dingo force-pushed the fix/normalizeProviders-flipflop-bug branch from af3acd6 to a80a21e Compare March 15, 2026 23:26
@slideshow-dingo slideshow-dingo marked this pull request as ready for review March 16, 2026 02:32
Comment on lines +112 to +118

// First normalization
normalizeProviders({ providers: originalProviders, agentDir });

// Simulate user editing models.json
const editedProviders: NonNullable<NonNullable<OpenClawConfig["models"]>["providers"]> = {
openai: {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Manual edits preserved" test does not chain the two normalizations

The first normalizeProviders call result is discarded (not assigned). The editedProviders object is then constructed from originalProviders directly and normalized in isolation. This correctly exercises the no-sourceProviders code path, but it does not cover the scenario described by the comment — where the output of a first normalization is manually edited and then re-normalized. A regression where the first pass mutates the value in a way that causes the second pass to revert it would go undetected.

For stronger coverage, consider passing the output of the first call as the base for the edit, so the test genuinely validates round-trip stability.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/models-config.providers.flipflop.test.ts
Line: 112-118

Comment:
**"Manual edits preserved" test does not chain the two normalizations**

The first `normalizeProviders` call result is discarded (not assigned). The `editedProviders` object is then constructed from `originalProviders` directly and normalized in isolation. This correctly exercises the no-`sourceProviders` code path, but it does not cover the scenario described by the comment — where the *output* of a first normalization is manually edited and then re-normalized. A regression where the first pass mutates the value in a way that causes the second pass to revert it would go undetected.

For stronger coverage, consider passing the output of the first call as the base for the edit, so the test genuinely validates round-trip stability.

How can I resolve this? If you propose a fix, please make it concise.

@slideshow-dingo
Copy link
Copy Markdown
Author

Greptile review comment fix - The third test 'FIX VERIFICATION: manual edits to models.json are preserved' had an issue where the first normalization result was discarded and edits were built from the original providers instead of the normalized output.

Fix applied:

-    // First normalization
-    normalizeProviders({ providers: originalProviders, agentDir });
+    // First normalization — capture the output
+    const normalized1 = normalizeProviders({ providers: originalProviders, agentDir });

-    // Simulate user editing models.json
+    // Simulate user editing the normalized output from models.json
     const editedProviders = {
       openai: {
-        ...originalProviders.openai,
-        apiKey: "edited-value",
+        ...normalized1!.openai!,
+        apiKey: "edited-value", // User manually edits after first normalization
       },
     };

All 4 tests pass with this fix.

Copy link
Copy Markdown
Author

@slideshow-dingo slideshow-dingo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "manual edits preserved" test should capture the first normalizeProviders result and use it as the base for editing. Currently editedProviders is built from originalProviders instead of the normalized output, so round-trip stability is not actually validated.

    // First normalization — capture the output
    const normalized1 = normalizeProviders({ providers: originalProviders, agentDir });

    // Simulate user editing the normalized output from models.json
    const editedProviders: NonNullable<NonNullable<OpenClawConfig["models"]>["providers"]> = {
      openai: {
        ...normalized1.openai!,
        apiKey: "edited-value", // User manually edits after first normalization
      },
    };

    // Second normalization: should preserve the manually edited value
    const normalized2 = normalizeProviders({ providers: editedProviders, agentDir });
    expect(normalized2?.openai?.apiKey).toBe("edited-value");

@slideshow-dingo
Copy link
Copy Markdown
Author

The "manual edits preserved" test should capture the first normalizeProviders result and use it as the base for editing. Currently editedProviders is built from originalProviders instead of the normalized output, so round-trip stability is not actually validated.

Fix:

// First normalization — capture the output
const normalized1 = normalizeProviders({ providers: originalProviders, agentDir });

// Simulate user editing the normalized output from models.json
const editedProviders = {
  openai: {
    ...normalized1.openai!,
    apiKey: "edited-value", // User manually edits after first normalization
  },
};

// Second normalization: should preserve the manually edited value
const normalized2 = normalizeProviders({ providers: editedProviders, agentDir });
expect(normalized2?.openai?.apiKey).toBe("edited-value");

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling channel: whatsapp-web Channel integration: whatsapp-web size: M

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant