sdk: centralize programmatic settings schema by neubig · Pull Request #2361 · OpenHands/software-agent-sdk

neubig · 2026-03-08T20:13:17Z

Summary

Adds the SDK-owned AgentSettings model plus export_schema(), export_settings_schema(), and create_agent() for programmatic agent configuration.
Uses the canonical LLM field definitions as the single source of truth for LLM settings metadata, while keeping the exported schema focused on structured configuration facts (sections, labels, defaults, dependencies, choices, secret-ness, and prominence).
Adds SDK coverage plus a runnable examples/01_standalone_sdk/46_agent_settings.py example for settings export, round-tripping, and settings-driven agent creation.
Keeps openhands-sdk at 1.14.0 on this non-release PR so version-bump and deprecation checks compare against the current released minor.

Companion GUI consumer PR: OpenHands/OpenHands#13306

Fixes #2228

Evidence

Local validation

✅ uv run pre-commit run --files AGENTS.md examples/01_standalone_sdk/46_agent_settings.py openhands-sdk/openhands/sdk/__init__.py openhands-sdk/openhands/sdk/llm/llm.py openhands-sdk/openhands/sdk/settings.py openhands-sdk/openhands/sdk/settings_metadata.py tests/sdk/test_settings.py
✅ VERSION_BUMP_BASE_REF=main PR_TITLE='sdk: centralize programmatic settings schema' PR_HEAD_REF='openhands/issue-2228-sdk-settings-schema' python3 .github/scripts/check_version_bumps.py
✅ uv run --with packaging python .github/scripts/check_deprecations.py
⚠️ ACP_VERSION_CHECK_BASE_REF=main ACP_VERSION_CHECK_SKIP=false uv run python .github/scripts/check_sdk_api_breakage.py currently reports the metadata-only Field(..., json_schema_extra=...) changes on public LLM fields as breaking changes.
✅ uv run pytest tests/sdk/test_settings.py -q (10 passed)

Live run

✅ LLM_API_KEY="$LLM_API_KEY" LLM_BASE_URL='https://llm-proxy.app.all-hands.dev' LLM_MODEL="$LLM_MODEL" uv run python examples/01_standalone_sdk/46_agent_settings.py
completed successfully, including the agent-created hello_settings.txt step and cleanup (EXAMPLE_COST: 0.032991).

Current GitHub status

❌ Python API breakage checks is currently red for the same metadata-only LLM field changes noted above.
❌ [Optional] Docs example/check-examples is currently red because examples/01_standalone_sdk/46_agent_settings.py is not yet documented in OpenHands/docs.
ℹ️ There is not currently a matching OpenHands/docs PR on branch openhands/issue-2228-sdk-settings-schema.

Checklist

If the PR is changing/adding functionality, are there tests to reflect this?
If there is an example, have you run the example to make sure that it works?
If there are instructions on how to run the code, have you followed the instructions and made sure that it works?
If the feature is significant enough to require documentation, is there a PR open on the OpenHands/docs repository with the same branch name?
Is the github CI passing?

AI note: this PR description was updated by an AI assistant (OpenHands) on behalf of the user.

Agent Server images for this PR

• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant	Architectures	Base Image	Docs / Tags
java	amd64, arm64	`eclipse-temurin:17-jdk`	Link
python	amd64, arm64	`nikolaik/python-nodejs:python3.13-nodejs22-slim`	Link
golang	amd64, arm64	`golang:1.21-bookworm`	Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:ca0d083-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-ca0d083-python \
  ghcr.io/openhands/agent-server:ca0d083-python

All tags pushed for this build

ghcr.io/openhands/agent-server:ca0d083-golang-amd64
ghcr.io/openhands/agent-server:ca0d083-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:ca0d083-golang-arm64
ghcr.io/openhands/agent-server:ca0d083-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:ca0d083-java-amd64
ghcr.io/openhands/agent-server:ca0d083-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:ca0d083-java-arm64
ghcr.io/openhands/agent-server:ca0d083-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:ca0d083-python-amd64
ghcr.io/openhands/agent-server:ca0d083-nikolaik_s_python-nodejs_tag_python3.13-nodejs22-slim-amd64
ghcr.io/openhands/agent-server:ca0d083-python-arm64
ghcr.io/openhands/agent-server:ca0d083-nikolaik_s_python-nodejs_tag_python3.13-nodejs22-slim-arm64
ghcr.io/openhands/agent-server:ca0d083-golang
ghcr.io/openhands/agent-server:ca0d083-java
ghcr.io/openhands/agent-server:ca0d083-python

About Multi-Architecture Support

Each variant tag (e.g., ca0d083-python) is a multi-arch manifest supporting both amd64 and arm64
Docker automatically pulls the correct architecture for your platform
Individual architecture tags (e.g., ca0d083-python-amd64) are also available if needed

Co-authored-by: openhands <openhands@all-hands.dev>

github-actions · 2026-03-08T20:13:42Z

Python API breakage checks — ✅ PASSED

Result: ✅ PASSED

Action log

github-actions · 2026-03-08T20:13:59Z

REST API breakage checks (OpenAPI) — ✅ PASSED

Result: ✅ PASSED

Action log

github-actions · 2026-03-08T20:15:35Z

Coverage Report •

File	Stmts	Miss	Cover	Missing
openhands-sdk/openhands/sdk
__init__.py	26	2	92%	95–96
openhands-sdk/openhands/sdk/llm
llm.py	481	78	83%	451, 468, 521, 749, 855, 857–858, 886, 932, 943–945, 949–953, 961–963, 973–975, 978–979, 983, 985–986, 988, 1186–1187, 1384–1385, 1394, 1407, 1409–1414, 1416–1433, 1436–1440, 1442–1443, 1449–1458, 1513, 1515
openhands-sdk/openhands/sdk/settings
model.py	250	15	94%	398, 423, 514, 519, 559, 591, 601, 603, 608, 626, 639, 641, 643, 645, 652
TOTAL	21244	6196	70%

all-hands-bot

Taste Rating: 🟢 Good taste

Analysis:

This PR solves a real problem - centralizing settings schema as a single source of truth for downstream clients. The data structure choice (Pydantic model with field metadata) is pragmatic and follows established patterns.

Data Structure: Clean. Settings live in one place, metadata is co-located with field definitions, and the schema export is straightforward.

Simplicity: The conversion logic handles real use cases (create vs update, custom factories) without over-engineering. Test coverage validates real behavior - agent round-tripping works, factories integrate correctly, schema export produces the right structure.

Backward Compatibility: Purely additive - no existing APIs changed.

Verdict: ✅ Worth merging

Key Insight: By making SDKSettings the declarative source of truth with exportable schema, this eliminates the need for clients to reverse-engineer agent configuration or maintain duplicate settings definitions.

Co-authored-by: openhands <openhands@all-hands.dev>

all-hands-bot

Taste Rating: 🟡 Acceptable - Solid engineering approach solving a real problem.

Data Structure: Clean. Settings metadata lives alongside field definitions via Pydantic annotations, avoiding the anti-pattern of duplicate config models that drift from the source of truth.

Verdict: ✅ Worth merging. The core design is sound - annotating canonical fields eliminates special cases and maintains a single source of truth. Minor documentation fix suggested below.

Co-authored-by: openhands <openhands@all-hands.dev>

enyst

Hi — OpenHands-GPT-5.4 here. I read the PR, the review threads, the related issues (#2228, #2274, #1451), and I also checked the current usage patterns in OpenHands/OpenHands-CLI before leaving this review.

I agree with the short-term motivation: programmatic settings should come from the SDK instead of being hand-curated separately in each client. I also think replacing the duplicate mini-LLM settings subset with the canonical LLM fields was the right local correction.

That said, I don't think this PR lands on the right boundary for the SDK yet.

The new AgentSettings round-trip is lossy. from_agent() / apply_to_agent() read like a general SDK config/edit/rebuild path, but the implementation only preserves a curated default story (LLMSummarizingCondenser, APIBasedCritic, and an annotated subset of LLM fields). In an extensible SDK, an official settings/materialization path should not silently drop alternative built-in or downstream implementations.
The exported schema mixes semantic config with client presentation policy. widget, advanced, placeholder, slash_command, and similar hints are first-party UX decisions, not core SDK semantics. Exporting them from SDK models hardcodes one client opinion into every client and makes future divergence painful.
More broadly, #2274 is right that LLM already carries too many responsibilities. This PR adds more meaning onto the current LLM class instead of separating configuration from runtime/materialization.

My preference would be either:

narrow this into an explicitly first-party client adapter (instead of a generic SDK settings abstraction), or
redesign around a canonical immutable config/profile object with neutral schema export, and keep presentation decisions in CLI/GUI.

So: useful motivation, but I think the current abstraction is risky for an extensible SDK.

Co-authored-by: openhands <openhands@all-hands.dev>

openhands-ai · 2026-03-27T04:13:22Z

I'm on it! neubig can track my progress at all-hands.dev

Co-authored-by: openhands <openhands@all-hands.dev>

neubig · 2026-03-27T04:17:40Z

@neubig Addressed in be4ff9a.

kept agent exported as the agent name, without changing create_agent() behavior
removed the exported required field from SettingsFieldSchema and from schema generation
exported tools in the general section so the schema matches the settings surface used by round-tripping and create_agent()

Validation:

uv run pre-commit run --files AGENTS.md openhands-sdk/openhands/sdk/settings.py tests/sdk/test_settings.py
uv run pytest tests/sdk/test_settings.py -q (12 passed)

openhands-ai · 2026-03-27T04:18:02Z

Here’s the final summary of the new work completed:

Addressed the requested PR feedback about settings schema behavior.
Kept the exported agent field as a descriptive agent name, without changing create_agent() behavior.
Removed the exported required field from SettingsFieldSchema and from schema generation.
Fixed schema export asymmetry by exporting tools in the general section.
Updated tests to match the new schema contract.
Updated AGENTS.md repository memory to reflect the new intended behavior.
Ran validation:
- uv run pre-commit run --files AGENTS.md openhands-sdk/openhands/sdk/settings.py tests/sdk/test_settings.py
- uv run pytest tests/sdk/test_settings.py -q → 12 passed
Committed and pushed the changes to the PR branch.
Replied on the PR with the update summary.

Conciseness check:

The code changes were minimal and focused on the review request.
No unrelated files were changed beyond the necessary code, tests, and repository memory update.
There are no obvious extraneous changes that should be reverted.

New commit:

be4ff9aa — fix(sdk): align settings schema export

PR reply:

sdk: centralize programmatic settings schema #2361 (comment)

So yes — the request has been completely addressed, and the follow-up was kept concise and faithful to the instructions.

xingyaoww

Overall LGTM! Would be helpful if we have a draft PR showing how CLI or GUI can use this refactored schema :)

I guess we can keep this feature hidden for a while and only publish the official agent settings example and docs when we've stabilize the API and got this support merged into our CLI & GUI

neubig · 2026-03-30T13:33:25Z

Yes, here are the draft PRs that are also waiting for review on the GUI+CLI. I haven't checked the CLI one very carefully yet:

@OpenHands perform the settings folder restructuring, per the review comment above.

openhands-ai · 2026-03-30T13:33:56Z

I'm on it! neubig can track my progress at all-hands.dev

Co-authored-by: openhands <openhands@all-hands.dev>

openhands-ai · 2026-03-30T13:41:59Z

Co-authored-by: openhands <openhands@all-hands.dev>

neubig · 2026-03-30T14:07:50Z

Thanks! Re-requested.

xingyaoww

LGTM! Let's get this in and try it in the CLI first -- Huge kudos for this!

xingyaoww · 2026-03-30T14:22:22Z

Related to #2361 (review)

@OpenHands can you create an issue tracking that we should add the settings example & docs ONCE we know it works in the CLI. Add a comment under OpenHands-CLI PR that @neubig mentioned and point it to the issue you created.

openhands-ai · 2026-03-30T14:22:47Z

I'm on it! xingyaoww can track my progress at all-hands.dev

openhands-ai · 2026-03-30T14:24:26Z

Summary

Both parts of the request have been fully addressed:

Created a tracking issue — OpenHands/software-agent-sdk#2606 to track adding the settings example & docs once the CLI integration is confirmed working. The issue references all relevant PRs (SDK sdk: centralize programmatic settings schema #2361, CLI Pillow now seems to be required in the agent server #580, GUI #13306) and the original issue Prepare settings for ingestion to programmatic settings menu in CLI and GUI #2228, and lists the blockers (CLI PR merged + API stabilization).
Commented on the CLI PR — OpenHands/OpenHands-CLI#580 with a pointer to the newly created tracking issue, so the CLI team has visibility.

No code changes were made — this was purely issue/comment management work, so there's nothing extraneous to revert.

sdk: centralize programmatic settings schema (#2228)

323059b

Co-authored-by: openhands <openhands@all-hands.dev>

all-hands-bot approved these changes Mar 8, 2026

View reviewed changes

Comment thread openhands-sdk/openhands/sdk/settings.py Outdated

sdk: fix settings schema required metadata

5356a26

Co-authored-by: openhands <openhands@all-hands.dev>

openhands-ai Bot mentioned this pull request Mar 8, 2026

Prepare settings for ingestion to programmatic settings menu in CLI and GUI #2228

Closed

Centralize agent settings metadata in the SDK

f075c6e

Co-authored-by: openhands <openhands@all-hands.dev>