Skip to content

Commit 0ef1f36

Browse files
Rob Riggsobviyus
Rob Riggs
authored and
obviyus
committed
feat(bedrock): add service_tier parameter support
- Add resolveBedrockServiceTier() and createBedrockServiceTierWrapper() to bedrock-stream-wrappers.ts - Export service tier functions from provider-stream-shared.ts SDK barrel - Wire service tier into Bedrock provider wrapStreamFn - Accepts serviceTier or service_tier via agents.defaults.params Valid values: default, flex, priority, reserved Authored by Deepseek-v4-Pro, reviewed by rob@mobilinkd.com.
1 parent a3e48fd commit 0ef1f36

4 files changed

Lines changed: 179 additions & 14 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ Docs: https://docs.openclaw.ai
77
### Changes
88

99
- Google/Gemini: normalize retired `google/gemini-3-pro-preview` and `google-gemini-cli/gemini-3-pro-preview` selections to `google/gemini-3.1-pro-preview` before they are written to model config.
10+
- Amazon Bedrock: support `serviceTier` parameter for Bedrock models, configurable via `agents.defaults.params.serviceTier` or per-model in `agents.defaults.models`. Valid values: `default`, `flex`, `priority`, `reserved`. (#64512) Thanks @mobilinkd.
1011
- Control UI: read the Quick Settings exec policy badge from `tools.exec.security` instead of the non-schema `agents.defaults.exec.security` path, so configured `full`/`deny` values render accurately. Fixes #78311. Thanks @FriedBack.
1112
- Control UI/usage: add transcript-backed historical lineage rollups for rotated logical sessions, with current-instance vs historical-lineage scope controls and long-range presets so usage history stays visible after restarts and updates. Fixes #50701. Thanks @dev-gideon-llc and @BunsDev.
1213
- Agents/failover: harden state-aware lane suspension by persisting quota resume transitions, restoring configured lane concurrency, preserving non-quota failure reasons, and exporting model failover events through diagnostics OTLP. Thanks @BunsDev.

‎docs/providers/bedrock.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -256,6 +256,49 @@ openclaw models list
256256

257257
</Accordion>
258258

259+
<Accordion title="Service tier">
260+
Some Bedrock models support a `service_tier` parameter to optimize for cost
261+
or latency. The following tiers are available:
262+
263+
| Tier | Description |
264+
|------|-------------|
265+
| `default` | Standard Bedrock tier |
266+
| `flex` | Discounted processing for workloads that can tolerate longer latency |
267+
| `priority` | Prioritized processing for latency-sensitive workloads |
268+
| `reserved` | Reserved capacity for steady-state workloads |
269+
270+
Set `serviceTier` (or `service_tier`) via `agents.defaults.params` for
271+
Bedrock model requests, or per-model in
272+
`agents.defaults.models["<model-key>"].params`:
273+
274+
```json5
275+
{
276+
agents: {
277+
defaults: {
278+
params: {
279+
serviceTier: "flex", // applies to all models
280+
},
281+
models: {
282+
"amazon-bedrock/mistral.mistral-large-3-675b-instruct": {
283+
params: {
284+
serviceTier: "priority", // per-model override
285+
},
286+
},
287+
},
288+
},
289+
},
290+
}
291+
```
292+
293+
Valid values are `default`, `flex`, `priority`, and `reserved`. Not all
294+
models support all tiers — if an unsupported tier is requested, Bedrock will
295+
return a validation error. Note: the error message is somewhat misleading;
296+
it may say "The provided model identifier is invalid" rather than indicating
297+
an unsupported service tier. If you see this error, check whether the model
298+
supports the requested tier.
299+
300+
</Accordion>
301+
259302
<Accordion title="Claude Opus 4.7 temperature">
260303
Bedrock rejects the `temperature` parameter for Claude Opus 4.7. OpenClaw
261304
omits `temperature` automatically for any Opus 4.7 Bedrock ref, including

‎extensions/amazon-bedrock/index.test.ts‎

Lines changed: 86 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -160,34 +160,28 @@ function makeAppInferenceProfileDescriptor(modelId: string): never {
160160
} as never;
161161
}
162162

163-
/**
164-
* Call wrapStreamFn and then invoke the returned stream function, capturing
165-
* the payload via the onPayload hook that streamWithPayloadPatch installs.
166-
*/
167163
async function callWrappedStream(
168164
provider: RegisteredProviderPlugin,
169165
modelId: string,
170166
modelDescriptor: never,
171167
config?: OpenClawConfig,
168+
extraParams?: Record<string, unknown>,
169+
payload: Record<string, unknown> = {},
172170
): Promise<Record<string, unknown>> {
173171
const wrapped = provider.wrapStreamFn?.({
174172
provider: "amazon-bedrock",
175173
modelId,
176174
config,
177175
streamFn: spyStreamFn,
176+
...(extraParams ? { extraParams } : {}),
178177
} as never);
179178

180-
// The wrapped stream returns the options object (from spyStreamFn).
181-
// For guardrail-wrapped streams, streamWithPayloadPatch intercepts onPayload,
182-
// so we need to invoke onPayload on the returned options to trigger the patch.
183179
const result = wrapped?.(modelDescriptor, { messages: [] } as never, {}) as unknown as Record<
184180
string,
185181
unknown
186182
>;
187183

188-
// If onPayload was installed by streamWithPayloadPatch, call it to apply the patch.
189184
if (typeof result?.onPayload === "function") {
190-
const payload: Record<string, unknown> = {};
191185
await (result.onPayload as (p: Record<string, unknown>, model: unknown) => Promise<unknown>)(
192186
payload,
193187
modelDescriptor,
@@ -719,6 +713,89 @@ describe("amazon-bedrock provider plugin", () => {
719713
});
720714
});
721715

716+
describe("service tier", () => {
717+
const CONVERSE_MODEL_DESCRIPTOR = {
718+
api: "bedrock-converse-stream",
719+
provider: "amazon-bedrock",
720+
id: NON_ANTHROPIC_MODEL,
721+
} as never;
722+
723+
it("injects serviceTier for valid camelCase value ('flex')", async () => {
724+
const provider = await registerWithConfig(undefined);
725+
const result = await callWrappedStream(
726+
provider,
727+
NON_ANTHROPIC_MODEL,
728+
CONVERSE_MODEL_DESCRIPTOR,
729+
runtimePluginConfig(undefined),
730+
{ serviceTier: "flex" },
731+
);
732+
expect(result._capturedPayload).toMatchObject({ serviceTier: { type: "flex" } });
733+
});
734+
735+
it("injects serviceTier for valid snake_case value ('priority')", async () => {
736+
const provider = await registerWithConfig(undefined);
737+
const result = await callWrappedStream(
738+
provider,
739+
NON_ANTHROPIC_MODEL,
740+
CONVERSE_MODEL_DESCRIPTOR,
741+
runtimePluginConfig(undefined),
742+
{ service_tier: "priority" },
743+
);
744+
expect(result._capturedPayload).toMatchObject({ serviceTier: { type: "priority" } });
745+
});
746+
747+
it("injects serviceTier for all valid tier names", async () => {
748+
const provider = await registerWithConfig(undefined);
749+
for (const tier of ["flex", "priority", "default", "reserved"] as const) {
750+
const result = await callWrappedStream(
751+
provider,
752+
NON_ANTHROPIC_MODEL,
753+
CONVERSE_MODEL_DESCRIPTOR,
754+
runtimePluginConfig(undefined),
755+
{ serviceTier: tier },
756+
);
757+
expect(result._capturedPayload).toMatchObject({ serviceTier: { type: tier } });
758+
}
759+
});
760+
761+
it("does not inject serviceTier when value is invalid", async () => {
762+
const provider = await registerWithConfig(undefined);
763+
const result = await callWrappedStream(
764+
provider,
765+
NON_ANTHROPIC_MODEL,
766+
CONVERSE_MODEL_DESCRIPTOR,
767+
runtimePluginConfig(undefined),
768+
{ serviceTier: "not-a-tier" },
769+
);
770+
expect(result).not.toHaveProperty("_capturedPayload");
771+
});
772+
773+
it("does not overwrite caller-provided serviceTier in payload", async () => {
774+
const provider = await registerWithConfig(undefined);
775+
const result = await callWrappedStream(
776+
provider,
777+
NON_ANTHROPIC_MODEL,
778+
CONVERSE_MODEL_DESCRIPTOR,
779+
runtimePluginConfig(undefined),
780+
{ serviceTier: "flex" },
781+
{ serviceTier: { type: "priority" } },
782+
);
783+
expect(result._capturedPayload).toMatchObject({ serviceTier: { type: "priority" } });
784+
});
785+
786+
it("skips injection for non-converse API models", async () => {
787+
const provider = await registerWithConfig(undefined);
788+
const result = await callWrappedStream(
789+
provider,
790+
NON_ANTHROPIC_MODEL,
791+
{ api: "openai-completions", provider: "amazon-bedrock", id: NON_ANTHROPIC_MODEL } as never,
792+
runtimePluginConfig(undefined),
793+
{ serviceTier: "flex" },
794+
);
795+
expect(result).not.toHaveProperty("_capturedPayload");
796+
});
797+
});
798+
722799
describe("application inference profile cache point injection", () => {
723800
/**
724801
* Invoke wrapStreamFn with a payload containing system/messages, then

‎extensions/amazon-bedrock/register.sync.runtime.ts‎

Lines changed: 49 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,43 @@ type AmazonBedrockPluginConfig = {
3434
guardrail?: GuardrailConfig;
3535
};
3636

37+
const BEDROCK_SERVICE_TIER_VALUES = ["flex", "priority", "default", "reserved"] as const;
38+
type BedrockServiceTier = (typeof BEDROCK_SERVICE_TIER_VALUES)[number];
39+
40+
function isBedrockServiceTier(value: string): value is BedrockServiceTier {
41+
return BEDROCK_SERVICE_TIER_VALUES.some((tier) => tier === value);
42+
}
43+
44+
function resolveBedrockServiceTier(
45+
extraParams: Record<string, unknown> | undefined,
46+
warn: (message: string) => void,
47+
): BedrockServiceTier | undefined {
48+
const raw = extraParams?.serviceTier ?? extraParams?.service_tier;
49+
if (typeof raw !== "string") {
50+
return undefined;
51+
}
52+
const normalized = raw.trim().toLowerCase();
53+
if (isBedrockServiceTier(normalized)) {
54+
return normalized;
55+
}
56+
warn(`ignoring invalid Bedrock service_tier param: ${raw}`);
57+
return undefined;
58+
}
59+
60+
function createBedrockServiceTierWrapper(
61+
underlying: StreamFn,
62+
serviceTier: BedrockServiceTier,
63+
): StreamFn {
64+
return (model, context, options) => {
65+
if (model.api !== "bedrock-converse-stream") {
66+
return underlying(model, context, options);
67+
}
68+
return streamWithPayloadPatch(underlying, model, context, options, (payloadObj) => {
69+
payloadObj.serviceTier ??= { type: serviceTier };
70+
});
71+
};
72+
}
73+
3774
function createGuardrailWrapStreamFn(
3875
innerWrapStreamFn: (ctx: { modelId: string; streamFn?: StreamFn }) => StreamFn | null | undefined,
3976
guardrailConfig: GuardrailConfig,
@@ -484,13 +521,20 @@ export function registerAmazonBedrockPlugin(api: OpenClawPluginApi): void {
484521
},
485522
resolveConfigApiKey: ({ env }) => resolveBedrockConfigApiKey(env),
486523
...anthropicByModelReplayHooks,
487-
wrapStreamFn: ({ modelId, config, model, streamFn, thinkingLevel }) => {
524+
wrapStreamFn: ({ modelId, config, model, streamFn, thinkingLevel, extraParams }) => {
488525
const currentGuardrail = resolveCurrentPluginConfig(config)?.guardrail;
489-
// Apply cache + guardrail wrapping.
490-
const wrapped =
491-
currentGuardrail?.guardrailIdentifier && currentGuardrail?.guardrailVersion
526+
let wrapped =
527+
(currentGuardrail?.guardrailIdentifier && currentGuardrail?.guardrailVersion
492528
? createGuardrailWrapStreamFn(baseWrapStreamFn, currentGuardrail)({ modelId, streamFn })
493-
: baseWrapStreamFn({ modelId, streamFn });
529+
: baseWrapStreamFn({ modelId, streamFn })) ?? undefined;
530+
531+
const serviceTier = resolveBedrockServiceTier(extraParams, (message) =>
532+
api.logger.warn(message),
533+
);
534+
if (serviceTier && wrapped) {
535+
wrapped = createBedrockServiceTierWrapper(wrapped, serviceTier);
536+
}
537+
494538
const region = resolveBedrockRegion(config) ?? extractRegionFromBaseUrl(model?.baseUrl);
495539
const mayNeedCacheInjection =
496540
isBedrockAppInferenceProfile(modelId) && !piAiWouldInjectCachePoints(modelId);

0 commit comments

Comments
 (0)