Fail closed when an explicit agent harness is missing (#71265)

pashpashpash · web-flow · commit 11804a484ded · 2026-04-25T06:39:57.000+09:00
* Fail closed for explicit agent harness selection

* Scope explicit harness fallback opt in
diff --git a/docs/.generated/config-baseline.sha256 b/docs/.generated/config-baseline.sha256
@@ -1,4 +1,4 @@
-bfae9b7760c3372f48d073da40059b0faa43c33f643b4aac3a942932a32df9eb  config-baseline.json
-c8ff25fcdd2389d5fd88f8ba188d77c21f58b56765b555eecf3b37437f743d50  config-baseline.core.json
+0adf332920764704575b21d2fe9568742d977ff0169683319c168d68ea7cf143  config-baseline.json
+2936d2ccf0c1e6e932a0e7c617b809e4b31dbb9a7d5afefbba29b229913b9e50  config-baseline.core.json
 22d7cd6d8279146b2d79c9531a55b80b52a2c99c81338c508104729154fdd02d  config-baseline.channel.json
-5bace1f246d5462dcf00ec7d4f378350bc7b6d01141609d704dc8c2e03e2230a  config-baseline.plugin.json
+28d874a4910174c7014ef2a267269a3327d31ff657f76d38c034ef1b86eae484  config-baseline.plugin.json
diff --git a/docs/gateway/config-agents.md b/docs/gateway/config-agents.md
@@ -369,7 +369,7 @@ Time format in system prompt. Default: `auto` (OS preference).
   - For direct OpenAI Responses models, server-side compaction is enabled automatically. Use `params.responsesServerCompaction: false` to stop injecting `context_management`, or `params.responsesCompactThreshold` to override the threshold. See [OpenAI server-side compaction](/providers/openai#server-side-compaction-responses-api).
 - `params`: global default provider parameters applied to all models. Set at `agents.defaults.params` (e.g. `{ cacheRetention: "long" }`).
 - `params` merge precedence (config): `agents.defaults.params` (global base) is overridden by `agents.defaults.models["provider/model"].params` (per-model), then `agents.list[].params` (matching agent id) overrides by key. See [Prompt Caching](/reference/prompt-caching) for details.
-- `embeddedHarness`: default low-level embedded agent runtime policy. Use `runtime: "auto"` to let registered plugin harnesses claim supported models, `runtime: "pi"` to force the built-in PI harness, or a registered harness id such as `runtime: "codex"`. Set `fallback: "none"` to disable automatic PI fallback. New Codex harness configs should keep model refs canonical as `openai/*` and select the harness here rather than using legacy `codex/*` model refs.
+- `embeddedHarness`: default low-level embedded agent runtime policy. Use `runtime: "auto"` to let registered plugin harnesses claim supported models, `runtime: "pi"` to force the built-in PI harness, or a registered harness id such as `runtime: "codex"`. Automatic PI fallback defaults to `"pi"` only in `auto` mode. Explicit plugin runtimes such as `codex` default to `"none"` unless you set `fallback: "pi"`. New Codex harness configs should keep model refs canonical as `openai/*` and select the harness here rather than using legacy `codex/*` model refs.
 - Config writers that mutate these fields (for example `/models set`, `/models set-image`, and fallback add/remove commands) save canonical object form and preserve existing fallback lists when possible.
 - `maxConcurrent`: max parallel agent runs across sessions (each session still serialized). Default: 4.
 
@@ -395,9 +395,9 @@ Codex app-server harness.
 ```
 
 - `runtime`: `"auto"`, `"pi"`, or a registered plugin harness id. The bundled Codex plugin registers `codex`.
-- `fallback`: `"pi"` or `"none"`. `"pi"` keeps the built-in PI harness as the compatibility fallback when no plugin harness is selected. `"none"` makes missing or unsupported plugin harness selection fail instead of silently using PI. Selected plugin harness failures always surface directly.
-- Environment overrides: `OPENCLAW_AGENT_RUNTIME=<id|auto|pi>` overrides `runtime`; `OPENCLAW_AGENT_HARNESS_FALLBACK=none` disables PI fallback for that process.
-- For Codex-only deployments, set `model: "openai/gpt-5.5"`, `embeddedHarness.runtime: "codex"`, and `embeddedHarness.fallback: "none"`.
+- `fallback`: `"pi"` or `"none"`. In `runtime: "auto"`, omitted fallback defaults to `"pi"` so old configs can keep using PI when no plugin harness claims a run. In explicit plugin runtime mode, such as `runtime: "codex"`, omitted fallback defaults to `"none"` so a missing harness fails instead of silently using PI. Runtime overrides do not inherit fallback from a broader scope; set `fallback: "pi"` alongside the explicit runtime when you intentionally want that compatibility fallback. Selected plugin harness failures always surface directly.
+- Environment overrides: `OPENCLAW_AGENT_RUNTIME=<id|auto|pi>` overrides `runtime`; `OPENCLAW_AGENT_HARNESS_FALLBACK=pi|none` overrides fallback for that process.
+- For Codex-only deployments, set `model: "openai/gpt-5.5"` and `embeddedHarness.runtime: "codex"`. You may also set `embeddedHarness.fallback: "none"` explicitly for readability; it is the default for explicit plugin runtimes.
 - Harness choice is pinned per session id after the first embedded run. Config/env changes affect new or reset sessions, not an existing transcript. Legacy sessions with transcript history but no recorded pin are treated as PI-pinned. `/status` shows non-PI harness ids such as `codex` next to `Fast`.
 - This only controls the embedded chat harness. Media generation, vision, PDF, music, video, and TTS still use their provider/model settings.
 
@@ -946,7 +946,7 @@ scripts/sandbox-browser-setup.sh   # optional browser image
 - `thinkingDefault`: optional per-agent default thinking level (`off | minimal | low | medium | high | xhigh | adaptive | max`). Overrides `agents.defaults.thinkingDefault` for this agent when no per-message or session override is set.
 - `reasoningDefault`: optional per-agent default reasoning visibility (`on | off | stream`). Applies when no per-message or session reasoning override is set.
 - `fastModeDefault`: optional per-agent default for fast mode (`true | false`). Applies when no per-message or session fast-mode override is set.
-- `embeddedHarness`: optional per-agent low-level harness policy override. Use `{ runtime: "codex", fallback: "none" }` to make one agent Codex-only while other agents keep the default PI fallback.
+- `embeddedHarness`: optional per-agent low-level harness policy override. Use `{ runtime: "codex" }` to make one agent Codex-only while other agents keep the default PI fallback in `auto` mode.
 - `runtime`: optional per-agent runtime descriptor. Use `type: "acp"` with `runtime.acp` defaults (`agent`, `backend`, `mode`, `cwd`) when the agent should default to ACP harness sessions.
 - `identity.avatar`: workspace-relative path, `http(s)` URL, or `data:` URI.
 - `identity` derives defaults: `ackReaction` from `emoji`, `mentionPatterns` from `name`/`emoji`.
diff --git a/docs/plugins/codex-harness.md b/docs/plugins/codex-harness.md
@@ -4,7 +4,7 @@ title: "Codex harness"
 read_when:
   - You want to use the bundled Codex app-server harness
   - You need Codex harness config examples
-  - You want to disable PI fallback for Codex-only deployments
+  - You want Codex-only deployments to fail instead of falling back to PI
 ---
 
 The bundled `codex` plugin lets OpenClaw run embedded agent turns through the
@@ -190,8 +190,9 @@ With this shape:
 
 ## Codex-only deployments
 
-Disable PI fallback when you need to prove that every embedded agent turn uses
-the Codex harness:
+Force the Codex harness when you need to prove that every embedded agent turn
+uses Codex. Explicit plugin runtimes default to no PI fallback, so
+`fallback: "none"` is optional but often useful as documentation:
 
 ```json5
 {
@@ -210,13 +211,13 @@ the Codex harness:
 Environment override:
 
 ```bash
-OPENCLAW_AGENT_RUNTIME=codex \
-OPENCLAW_AGENT_HARNESS_FALLBACK=none \
-openclaw gateway run
+OPENCLAW_AGENT_RUNTIME=codex openclaw gateway run
 ```
 
-With fallback disabled, OpenClaw fails early if the Codex plugin is disabled,
-the app-server is too old, or the app-server cannot start.
+With Codex forced, OpenClaw fails early if the Codex plugin is disabled, the
+app-server is too old, or the app-server cannot start. Set
+`OPENCLAW_AGENT_HARNESS_FALLBACK=pi` only if you intentionally want PI to handle
+missing harness selection.
 
 ## Per-agent Codex
 
@@ -581,12 +582,12 @@ understanding continue to use the matching provider/model settings such as
 select an `openai/gpt-*` model with `embeddedHarness.runtime: "codex"` (or a
 legacy `codex/*` ref), and check whether `plugins.allow` excludes `codex`.
 
-**OpenClaw uses PI instead of Codex:** if no Codex harness claims the run,
-OpenClaw may use PI as the compatibility backend. Set
-`embeddedHarness.runtime: "codex"` to force Codex selection while testing, or
-`embeddedHarness.fallback: "none"` to fail when no plugin harness matches. Once
-Codex app-server is selected, its failures surface directly without extra
-fallback config.
+**OpenClaw uses PI instead of Codex:** `runtime: "auto"` can still use PI as the
+compatibility backend when no Codex harness claims the run. Set
+`embeddedHarness.runtime: "codex"` to force Codex selection while testing. A
+forced Codex runtime now fails instead of falling back to PI unless you
+explicitly set `embeddedHarness.fallback: "pi"`. Once Codex app-server is
+selected, its failures surface directly without extra fallback config.
 
 **The app-server is rejected:** upgrade Codex so the app-server handshake
 reports version `0.118.0` or newer.
diff --git a/src/agents/harness/selection.test.ts b/src/agents/harness/selection.test.ts
@@ -100,15 +100,36 @@ function registerFailingCodexHarness(): void {
 }
 
 describe("runAgentHarnessAttemptWithFallback", () => {
-  it("falls back to the PI harness when a forced plugin harness is unavailable", async () => {
+  it("fails when a forced plugin harness is unavailable and fallback is omitted", async () => {
     process.env.OPENCLAW_AGENT_RUNTIME = "codex";
 
+    await expect(runAgentHarnessAttemptWithFallback(createAttemptParams())).rejects.toThrow(
+      'Requested agent harness "codex" is not registered and PI fallback is disabled.',
+    );
+    expect(piRunAttempt).not.toHaveBeenCalled();
+  });
+
+  it("falls back to the PI harness for a forced plugin harness only when explicitly configured", async () => {
+    process.env.OPENCLAW_AGENT_RUNTIME = "codex";
+    process.env.OPENCLAW_AGENT_HARNESS_FALLBACK = "pi";
+
     const result = await runAgentHarnessAttemptWithFallback(createAttemptParams());
 
     expect(result.sessionIdUsed).toBe("pi");
     expect(piRunAttempt).toHaveBeenCalledTimes(1);
   });
 
+  it("does not inherit config fallback when env forces a plugin harness", async () => {
+    process.env.OPENCLAW_AGENT_RUNTIME = "codex";
+
+    await expect(
+      runAgentHarnessAttemptWithFallback(
+        createAttemptParams({ agents: { defaults: { embeddedHarness: { fallback: "pi" } } } }),
+      ),
+    ).rejects.toThrow('Requested agent harness "codex" is not registered');
+    expect(piRunAttempt).not.toHaveBeenCalled();
+  });
+
   it("falls back to the PI harness in auto mode when no plugin harness matches", async () => {
     process.env.OPENCLAW_AGENT_RUNTIME = "auto";
 
@@ -177,6 +198,56 @@ describe("runAgentHarnessAttemptWithFallback", () => {
     ).rejects.toThrow("PI fallback is disabled");
     expect(piRunAttempt).not.toHaveBeenCalled();
   });
+
+  it("fails for config-forced plugin harnesses when fallback is omitted", async () => {
+    await expect(
+      runAgentHarnessAttemptWithFallback(
+        createAttemptParams({ agents: { defaults: { embeddedHarness: { runtime: "codex" } } } }),
+      ),
+    ).rejects.toThrow('Requested agent harness "codex" is not registered');
+    expect(piRunAttempt).not.toHaveBeenCalled();
+  });
+
+  it("allows config-forced plugin harnesses to opt into PI fallback", async () => {
+    const result = await runAgentHarnessAttemptWithFallback(
+      createAttemptParams({
+        agents: { defaults: { embeddedHarness: { runtime: "codex", fallback: "pi" } } },
+      }),
+    );
+
+    expect(result.sessionIdUsed).toBe("pi");
+    expect(piRunAttempt).toHaveBeenCalledTimes(1);
+  });
+
+  it("does not inherit default fallback when an agent forces a plugin harness", async () => {
+    await expect(
+      runAgentHarnessAttemptWithFallback({
+        ...createAttemptParams({
+          agents: {
+            defaults: { embeddedHarness: { fallback: "pi" } },
+            list: [{ id: "strict", embeddedHarness: { runtime: "codex" } }],
+          },
+        }),
+        sessionKey: "agent:strict:session-1",
+      }),
+    ).rejects.toThrow('Requested agent harness "codex" is not registered');
+    expect(piRunAttempt).not.toHaveBeenCalled();
+  });
+
+  it("lets an agent-forced plugin harness opt into PI fallback", async () => {
+    const result = await runAgentHarnessAttemptWithFallback({
+      ...createAttemptParams({
+        agents: {
+          defaults: { embeddedHarness: { fallback: "none" } },
+          list: [{ id: "strict", embeddedHarness: { runtime: "codex", fallback: "pi" } }],
+        },
+      }),
+      sessionKey: "agent:strict:session-1",
+    });
+
+    expect(result.sessionIdUsed).toBe("pi");
+    expect(piRunAttempt).toHaveBeenCalledTimes(1);
+  });
 });
 
 describe("selectAgentHarness", () => {
diff --git a/src/agents/harness/selection.ts b/src/agents/harness/selection.ts
@@ -333,12 +333,45 @@ export function resolveAgentHarnessPolicy(params: {
     : normalizeEmbeddedAgentRuntime(agentPolicy?.runtime ?? defaultsPolicy?.runtime);
   return {
     runtime,
-    fallback:
-      resolveEmbeddedAgentHarnessFallback(env) ??
-      normalizeAgentHarnessFallback(agentPolicy?.fallback ?? defaultsPolicy?.fallback),
+    fallback: resolveAgentHarnessFallbackPolicy({
+      env,
+      runtime,
+      agentPolicy,
+      defaultsPolicy,
+    }),
   };
 }
 
+function resolveAgentHarnessFallbackPolicy(params: {
+  env: NodeJS.ProcessEnv;
+  runtime: EmbeddedAgentRuntime;
+  agentPolicy?: AgentEmbeddedHarnessConfig;
+  defaultsPolicy?: AgentEmbeddedHarnessConfig;
+}): EmbeddedAgentHarnessFallback {
+  const envFallback = resolveEmbeddedAgentHarnessFallback(params.env);
+  if (envFallback) {
+    return envFallback;
+  }
+
+  const envRuntime = params.env.OPENCLAW_AGENT_RUNTIME?.trim();
+  if (envRuntime && isPluginAgentRuntime(params.runtime)) {
+    return normalizeAgentHarnessFallback(undefined, params.runtime);
+  }
+
+  if (params.agentPolicy?.runtime) {
+    return normalizeAgentHarnessFallback(params.agentPolicy.fallback, params.runtime);
+  }
+
+  return normalizeAgentHarnessFallback(
+    params.agentPolicy?.fallback ?? params.defaultsPolicy?.fallback,
+    params.runtime,
+  );
+}
+
+function isPluginAgentRuntime(runtime: EmbeddedAgentRuntime): boolean {
+  return runtime !== "auto" && runtime !== "pi";
+}
+
 function resolveAgentEmbeddedHarnessConfig(
   config: OpenClawConfig | undefined,
   params: { agentId?: string; sessionKey?: string },
@@ -357,8 +390,12 @@ function resolveAgentEmbeddedHarnessConfig(
 
 function normalizeAgentHarnessFallback(
   value: AgentEmbeddedHarnessConfig["fallback"] | undefined,
+  runtime: EmbeddedAgentRuntime,
 ): EmbeddedAgentHarnessFallback {
-  return value === "none" ? "none" : "pi";
+  if (value) {
+    return value === "none" ? "none" : "pi";
+  }
+  return runtime === "auto" ? "pi" : "none";
 }
 
 function formatProviderModel(params: { provider: string; modelId?: string }): string {
diff --git a/src/config/schema.base.generated.ts b/src/config/schema.base.generated.ts
@@ -3038,7 +3038,7 @@ export const GENERATED_BASE_CONFIG_SCHEMA: BaseConfigSchemaResponse = {
                     enum: ["pi", "none"],
                     title: "Default Embedded Harness Fallback",
                     description:
-                      "Embedded harness fallback when no plugin harness matches. Selected plugin harness failures surface directly. Set none to disable automatic PI fallback.",
+                      "Embedded harness fallback when no plugin harness matches. Auto mode defaults to pi; explicit plugin runtimes default to none and do not inherit broader fallback settings. Selected plugin harness failures surface directly.",
                   },
                 },
                 additionalProperties: false,
@@ -5793,13 +5793,13 @@ export const GENERATED_BASE_CONFIG_SCHEMA: BaseConfigSchemaResponse = {
                       enum: ["pi", "none"],
                       title: "Agent Embedded Harness Fallback",
                       description:
-                        "Per-agent embedded harness fallback. Set none to disable automatic PI fallback for this agent.",
+                        "Per-agent embedded harness fallback. Auto mode defaults to pi; explicit plugin runtimes default to none and do not inherit broader fallback settings.",
                     },
                   },
                   additionalProperties: false,
                   title: "Agent Embedded Harness",
                   description:
-                    "Per-agent embedded harness policy override. Use fallback=none to make missing plugin harness selection fail instead of falling back to PI.",
+                    "Per-agent embedded harness policy override. Use runtime=codex to force Codex for one agent while defaults stay in auto mode.",
                 },
                 model: {
                   anyOf: [
@@ -23513,7 +23513,7 @@ export const GENERATED_BASE_CONFIG_SCHEMA: BaseConfigSchemaResponse = {
     },
     "agents.defaults.embeddedHarness.fallback": {
       label: "Default Embedded Harness Fallback",
-      help: "Embedded harness fallback when no plugin harness matches. Selected plugin harness failures surface directly. Set none to disable automatic PI fallback.",
+      help: "Embedded harness fallback when no plugin harness matches. Auto mode defaults to pi; explicit plugin runtimes default to none and do not inherit broader fallback settings. Selected plugin harness failures surface directly.",
       tags: ["reliability"],
     },
     "agents.list": {
@@ -23558,7 +23558,7 @@ export const GENERATED_BASE_CONFIG_SCHEMA: BaseConfigSchemaResponse = {
     },
     "agents.list.*.embeddedHarness": {
       label: "Agent Embedded Harness",
-      help: "Per-agent embedded harness policy override. Use fallback=none to make missing plugin harness selection fail instead of falling back to PI.",
+      help: "Per-agent embedded harness policy override. Use runtime=codex to force Codex for one agent while defaults stay in auto mode.",
       tags: ["advanced"],
     },
     "agents.list.*.embeddedHarness.runtime": {
@@ -23568,7 +23568,7 @@ export const GENERATED_BASE_CONFIG_SCHEMA: BaseConfigSchemaResponse = {
     },
     "agents.list.*.embeddedHarness.fallback": {
       label: "Agent Embedded Harness Fallback",
-      help: "Per-agent embedded harness fallback. Set none to disable automatic PI fallback for this agent.",
+      help: "Per-agent embedded harness fallback. Auto mode defaults to pi; explicit plugin runtimes default to none and do not inherit broader fallback settings.",
       tags: ["reliability"],
     },
     "gateway.port": {
diff --git a/src/config/schema.help.ts b/src/config/schema.help.ts
@@ -1155,13 +1155,13 @@ export const FIELD_HELP: Record<string, string> = {
   "agents.defaults.embeddedHarness.runtime":
     "Embedded harness runtime: auto, pi, or a registered plugin harness id such as codex.",
   "agents.defaults.embeddedHarness.fallback":
-    "Embedded harness fallback when no plugin harness matches. Selected plugin harness failures surface directly. Set none to disable automatic PI fallback.",
+    "Embedded harness fallback when no plugin harness matches. Auto mode defaults to pi; explicit plugin runtimes default to none and do not inherit broader fallback settings. Selected plugin harness failures surface directly.",
   "agents.list.*.embeddedHarness":
-    "Per-agent embedded harness policy override. Use fallback=none to make missing plugin harness selection fail instead of falling back to PI.",
+    "Per-agent embedded harness policy override. Use runtime=codex to force Codex for one agent while defaults stay in auto mode.",
   "agents.list.*.embeddedHarness.runtime":
     "Per-agent embedded harness runtime: auto, pi, or a registered plugin harness id such as codex.",
   "agents.list.*.embeddedHarness.fallback":
-    "Per-agent embedded harness fallback. Set none to disable automatic PI fallback for this agent.",
+    "Per-agent embedded harness fallback. Auto mode defaults to pi; explicit plugin runtimes default to none and do not inherit broader fallback settings.",
   "agents.defaults.imageModel.primary":
     "Optional image model (provider/model) used when the primary model lacks image input.",
   "agents.defaults.imageModel.fallbacks": "Ordered fallback image models (provider/model).",
