Skip to content

settings: expose SDK settings schema to OpenHands#13306

Merged
tofarr merged 241 commits intomainfrom
openhands/issue-2228-gui-settings-schema
Apr 15, 2026
Merged

settings: expose SDK settings schema to OpenHands#13306
tofarr merged 241 commits intomainfrom
openhands/issue-2228-gui-settings-schema

Conversation

@neubig
Copy link
Copy Markdown
Contributor

@neubig neubig commented Mar 8, 2026

What

Integrates the SDK-owned AgentSettings schema into the OpenHands GUI and backend. The frontend now fetches the schema from /api/settings/schema (served by the SDK) and dynamically renders settings pages from it, instead of hard-coding field definitions.

The most important points are:

  1. The settings menus can be dynamically created from AgentSettings, so updates to the SDK agent settings will immediately be exposed to users.
  2. These settings will be plumbed all the way through to a conversation start request, so new conversations will use these settings

Fixes #13788 · Upstream: software-agent-sdk#2228

NOTE: This PR also modifies v0 code, as this was the easiest way to ensure that the entire codebase was consistent. Feel free to skip review of the v0 code, as it will be removed soon anyway.

Why

The SDK defines the AgentSettings model, so it should own the schema contract. This eliminates manual duplication of settings fields in the frontend and ensures new SDK settings appear in the GUI automatically.

Key changes

Backend — Settings model (openhands/storage/data_models/settings.py)

  • Settings now embeds AgentSettings (from SDK) and ConversationSettings as nested objects
  • Flat legacy fields (llm_model, llm_api_key, llm_base_url, agent, etc.) are migrated into agent_settings / conversation_settings on load
  • Enterprise DB migration adds agent_settings column (migration 107)

Backend — MCP config (openhands/core/config/mcp_config.py, openhands/mcp/)

  • MCPConfig and MCPStdioServerConfig now come from the SDK
  • Removed include_stdio / use_stdio parameters from create_mcp_clients and fetch_mcp_tools_from_config — all configured servers are always connected

Backend — Conversation startup

  • live_status_app_conversation_service.py: Reads agent/conversation settings from AgentSettings; renamed _create_condenser_create_llm_summarizing_condenser
  • session.py, conversation_service.py: Adapted to pass AgentSettings through startup flow

Frontend — Schema-driven settings UI

  • New: sdk-section-page.tsx, schema-field.tsx — generic components that render any SDK settings section from schema JSON
  • New: use-agent-settings-schema.ts hook fetches /api/settings/schema
  • New: sdk-settings-schema.ts — utilities to parse schema sections, extract field metadata, build value pickers
  • New routes: /settings/condenser, /settings/verification, plus org-default variants
  • llm-settings.tsx: Rewritten to render from schema with custom overrides for model selector
  • mcp-settings.tsx: Adapted to use MCPConfig from SDK types
  • use-settings.ts: Now loads/saves agent_settings and conversation_settings as nested objects
  • use-save-settings.ts: Handles JSON-patch merging of nested settings

Enterprise

  • saas_settings_store.py, org_store.py, org_member_store.py, user_store.py: Adapted to AgentSettings-based model
  • org_models.py: Simplified — org defaults use AgentSettings directly
  • DB migration 107: Adds agent_settings JSON column to enterprise settings table

SDK dependency

  • Pinned to software-agent-sdk@5aef1b2 (includes schema route + AgentSettings / ConversationSettings models)
  • Uses AGENT_SETTINGS_SCHEMA_VERSION constant from SDK instead of hardcoded values

How to review

This is a large PR (~20k lines across 137 files). Suggested review order:

  1. Data modelopenhands/storage/data_models/settings.py (the core change)
  2. Settings routeropenhands/server/routes/settings.py and openhands/app_server/settings/settings_router.py
  3. Frontend schema utilitiesfrontend/src/utils/sdk-settings-schema.ts
  4. Frontend settings pagesfrontend/src/routes/llm-settings.tsx, sdk-section-page.tsx
  5. MCP changesopenhands/core/config/mcp_config.py, openhands/mcp/utils.py
  6. Enterprise storageenterprise/storage/saas_settings_store.py, enterprise/storage/org_store.py
  7. Tests — verify coverage of new paths

This PR description was written by an AI assistant (OpenHands) on behalf of the user.


To run this PR locally, use the following command:

GUI with Docker:

docker run -it --rm   -p 3000:3000   -v /var/run/docker.sock:/var/run/docker.sock   --add-host host.docker.internal:host-gateway   -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.openhands.dev/openhands/runtime:d1926a9-nikolaik   --name openhands-app-d1926a9   docker.openhands.dev/openhands/openhands:d1926a9

Co-authored-by: openhands <openhands@all-hands.dev>
Copy link
Copy Markdown
Collaborator

@all-hands-bot all-hands-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Acceptable - Clean implementation that solves a real problem, but testing needs strengthening.

The core logic is sound: graceful SDK detection with clean try/except, proper type definitions, and backward-compatible API extension. However, the test coverage is too shallow for a feature explicitly designed to handle SDK presence/absence.

Key Issues:

  1. Only one field (model_name) is verified in the test - schema structure is not validated
  2. Missing test for the None case when SDK is not installed (contradicts the graceful degradation design)
  3. Missing docstring on the helper function

Suggested Addition:
Add a test case for when SDK is missing (e.g., mock the import to raise ModuleNotFoundError and verify sdk_settings_schema is None).

Comment thread tests/unit/server/routes/test_settings_api.py Outdated
Comment thread openhands/server/routes/settings.py
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
@neubig neubig requested a review from chuckbutkus as a code owner March 8, 2026 21:09
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Mar 8, 2026

Coverage report

Warning

The diff for this PR is too large to be retrieved from GitHub's API (maximum 300 files). Diff coverage is not available for this PR.

This PR does not seem to contain any modification to coverable code.

@neubig neubig force-pushed the openhands/issue-2228-gui-settings-schema branch from d87179d to 11a0803 Compare March 8, 2026 21:38
Co-authored-by: openhands <openhands@all-hands.dev>
@neubig neubig force-pushed the openhands/issue-2228-gui-settings-schema branch from 11a0803 to 9dab5b1 Compare March 8, 2026 21:43
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
@neubig neubig marked this pull request as draft March 9, 2026 03:07
Co-authored-by: openhands <openhands@all-hands.dev>
xingyaoww pushed a commit to OpenHands/software-agent-sdk that referenced this pull request Mar 13, 2026
Add methods to OpenHandsCloudWorkspace that call the new SaaS API
endpoints to retrieve the user's LLM configuration and custom secrets:

- get_llm(**llm_kwargs): Fetches LLM settings from the user's SaaS
  account and returns a configured LLM instance. User kwargs override
  SaaS defaults.
- get_secrets(names=None): Fetches custom secrets and returns a
  dict[str, str] compatible with conversation.update_secrets().

These methods enable SDK users to inherit their SaaS credentials while
retaining full control over agent customization.

Depends on OpenHands/OpenHands#13306 for the server-side API endpoints.

Related: OpenHands/OpenHands#13268

Co-authored-by: openhands <openhands@all-hands.dev>
@neubig
Copy link
Copy Markdown
Contributor Author

neubig commented Mar 13, 2026

@OpenHands merge main and fix conflicts

then update the code in this PR to reflect the most recent changes here: OpenHands/software-agent-sdk#2361

respond with instructions about how I can test this locally

@openhands-ai
Copy link
Copy Markdown

openhands-ai Bot commented Mar 13, 2026

I'm on it! neubig can track my progress at all-hands.dev

Co-authored-by: openhands <openhands@all-hands.dev>
Copy link
Copy Markdown
Contributor Author

neubig commented Mar 13, 2026

Merged main into this branch and updated the settings/schema integration to match the latest SDK changes from software-agent-sdk#2361. The branch now includes commit 918c44d16.

Local verification steps:

  1. Sync the branch and install dependencies:

    • git checkout openhands/issue-2228-gui-settings-schema
    • git pull
    • poetry install
    • cd frontend && npm install && cd ..
  2. Run the targeted backend tests:

    • poetry run pytest tests/unit/server/routes/test_settings_api.py tests/unit/storage/data_models/test_settings.py tests/unit/app_server/test_live_status_app_conversation_service.py -q
  3. Run the targeted frontend tests:

    • cd frontend && npm run test -- src/utils/sdk-settings-schema.test.ts __tests__/routes/llm-settings.test.tsx
  4. Re-run the validation checks I used before pushing:

    • poetry run pre-commit run --config ./dev_config/python/.pre-commit-config.yaml --files openhands/server/routes/settings.py openhands/storage/data_models/settings.py openhands/app_server/app_conversation/live_status_app_conversation_service.py tests/unit/server/routes/test_settings_api.py tests/unit/storage/data_models/test_settings.py tests/unit/app_server/test_live_status_app_conversation_service.py pyproject.toml poetry.lock enterprise/poetry.lock uv.lock
    • cd frontend && npm run lint:fix && npm run build
  5. Optional manual smoke test:

    • run the app, open Settings → LLM, confirm the SDK-backed schema renders with basic/advanced sections, save changes, and refresh to verify dotted sdk_settings_values persist correctly (including structured JSON/object values when present in the schema).

@openhands-ai
Copy link
Copy Markdown

openhands-ai Bot commented Mar 13, 2026

Final update:

Checklist

  • Merged main into openhands/issue-2228-gui-settings-schema
  • Updated the PR implementation to match the latest SDK schema changes from software-agent-sdk#2361
  • Kept the changes focused on the settings-schema integration plus required regenerated lockfiles
  • Passed targeted backend tests, frontend tests, frontend build, and backend pre-commit checks
  • Pushed the final changes to the remote PR branch

New/final changes since the last summary

  • Fixed a frontend type issue in frontend/src/mocks/settings-handlers.ts that was blocking the pre-commit/typecheck path during commit.
  • Successfully committed the final merged/update work as:
    • 918c44d16Merge main and align settings schema with latest SDK
  • Pushed that commit to:
    • origin/openhands/issue-2228-gui-settings-schema
  • Cleaned up the accidentally malformed PR comment and left the corrected testing instructions comment in place:

Final result

The PR branch is now updated, pushed, and includes the requested merge-from-main plus the SDK-settings-schema adaptations, along with local verification instructions for the reviewer.

@neubig
Copy link
Copy Markdown
Contributor Author

neubig commented Mar 15, 2026

@OpenHands merge main and fix failing CI

@openhands-ai
Copy link
Copy Markdown

openhands-ai Bot commented Mar 15, 2026

I'm on it! neubig can track my progress at all-hands.dev

Co-authored-by: openhands <openhands@all-hands.dev>
Copy link
Copy Markdown
Contributor Author

neubig commented Mar 15, 2026

Merged current main into this branch and fixed the failing Docker build by regenerating poetry.lock, enterprise/poetry.lock, and uv.lock against the SDK branch refs required by this PR.

Local verification completed:

  • poetry install --no-root
  • poetry run pytest tests/unit/server/routes/test_settings_api.py tests/unit/storage/data_models/test_settings.py tests/unit/app_server/test_live_status_app_conversation_service.py -q
  • poetry run pre-commit run --config ./dev_config/python/.pre-commit-config.yaml --files pyproject.toml poetry.lock enterprise/poetry.lock uv.lock openhands/server/routes/settings.py tests/unit/server/routes/test_settings_api.py
  • cd frontend && npm run test -- __tests__/routes/llm-settings.test.tsx
  • cd frontend && npm run lint:fix && npm run build
  • cd enterprise && poetry install --with dev,test --no-root && PYTHONPATH=".:$PYTHONPATH" poetry run pytest tests/unit --collect-only -q

Pushed in 19a089aa4, so CI should be rerunning on the updated branch now.

@neubig
Copy link
Copy Markdown
Contributor Author

neubig commented Mar 16, 2026

@OpenHands merge main and fix any issues. make sure that this runs appropriately.

@openhands-ai
Copy link
Copy Markdown

openhands-ai Bot commented Mar 16, 2026

I'm on it! neubig can track my progress at all-hands.dev

Co-authored-by: openhands <openhands@all-hands.dev>
@hieptl
Copy link
Copy Markdown
Collaborator

hieptl commented Apr 15, 2026

Hello @neubig,

I encountered a few issues while running the pull request locally in OSS mode. I’ve outlined them below for your reference.

Issue 1 (Blocking): The width of the input elements is inconsistent. Please see the image below for further details.

issue1

Issue 2 (Blocking): The Basic View of the "Verification" page is currently empty. You can refer to the image below for clarification.

issue2

Issue 3 (Blocking): After adding MCP, I’m unable to delete the added data. A video has been included below for additional context.

issue3.mov

Issue 4 (Blocking): The agent dropdown is appearing empty. Kindly refer to the image below for further information.

issue4

Issue 5 (Blocking): Is the link supposed to be https://docs.openhands.dev? Please see the image below for further details.

issue5

Issue 6 (Blocking): When attempting to set the value "99999" in the Top P field and clicking the Save Settings button, I receive the backend error message 'Something went wrong storing settings.' If users input values across multiple fields, it may lead to confusion, as it will not be clear which fields are invalid. For further details, please refer to the image below.

issue6

Issue 7 (Blocking): While most new fields are accompanied by helpful explanatory texts placed below the input elements, some fields are missing these descriptions. A video has been included below for further clarification.

issue7.mov

Issue 8 (Blocking): It seems that multiple languages are not currently supported for the new fields. You can refer to the video below for additional context.

issue8.mov

Thank you very much! 🙏

secret_store = secret_store.model_copy(
update={'custom_secrets': converted_store.custom_secrets}
)
data['secret_store'] = secret_store
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion (blocking):

data['secret_store'] = secret_store # the field is actually named 'secrets_store' (plural).

It appears that the field in question is secrets_store (as seen on line 106). However, the validator is attempting to write to a key that does not exist.

Thank you! 🙏

… in settings round-trip

The SDK's AgentSettings.model_validate automatically transforms
'openhands/X' to 'litellm_proxy/X' and sets the proxy base_url.
This caused three issues:

1. is_openhands_model() only recognized 'openhands/' prefix, so
   _post_merge_llm_fixups and the V1 load_settings base_url clearing
   logic failed for SDK-transformed model names.

2. get_agent_settings_display() converted the model name back but
   didn't clear the proxy base_url, so the frontend received the raw
   proxy URL and could not display 'basic' mode.

3. The V1 load_settings endpoint didn't convert the model name back
   to 'openhands/' and had a trailing-slash mismatch when comparing
   base_url to LITE_LLM_API_URL.

Fixes:
- is_openhands_model() now recognizes both 'openhands/' and
  'litellm_proxy/' prefixes
- get_agent_settings_display() clears the proxy base_url for managed
  models
- V1 load_settings converts litellm_proxy/ → openhands/ and normalizes
  trailing slashes in the base_url comparison

Co-authored-by: openhands <openhands@all-hands.dev>
Copy link
Copy Markdown
Contributor Author

neubig commented Apr 15, 2026

Pushed 0caab0a to address the valid settings review regressions.

Added/expanded regression coverage in:

  • frontend/__tests__/components/modals/settings/model-selector.test.tsx
  • frontend/__tests__/routes/llm-settings.test.tsx
  • frontend/__tests__/routes/mcp-settings.test.tsx
  • frontend/__tests__/routes/verification-settings.test.tsx
  • frontend/__tests__/components/features/settings/sdk-settings/schema-field.test.tsx
  • frontend/__tests__/settings-schema-descriptions.test.ts
  • tests/unit/storage/data_models/test_settings.py

These tests cover the review issues around schema-driven labels/descriptions/constraints, basic-view LLM + verification behavior, MCP deletion/replacement, and preserving current agent selections.

Validation summary:

  • Pre-fix: the new regression cases failed against the prior branch state when they were introduced.
  • Post-fix: targeted frontend regressions now pass locally: 56/56 tests passing across the six settings-focused Vitest files above.
  • Backend: added focused unit coverage for MCP replacement/clearing in test_settings.py, and syntax-checked the modified Python files with python -m py_compile. Full backend pytest execution was not available in this container because the local Python environment is missing repo test dependencies.

No SDK-side changes were required for these fixes.

This comment was created by an AI assistant (OpenHands) on behalf of the user.

tofarr and others added 13 commits April 15, 2026 09:31
Implements Rails-style i18n conventions for schema-driven UI fields:

- Add toSchemaTranslationKey() to generate conventional keys
  (e.g., llm.api_key -> SCHEMA$LLM$API_KEY$LABEL)
- Add resolver functions with fallback chain:
  - resolveSchemaFieldLabel()
  - resolveSchemaFieldDescription()
  - resolveSchemaFieldSectionLabel()
  - resolveSchemaChoiceLabel()
- Update schema-field.tsx to use new resolver functions
- Add ~18 SCHEMA translation keys for common fields with all 15 languages
- Add 20 unit tests for resolver functions

The fallback chain:
1. Use explicit translation key if schema provides one (contains $)
2. Try conventional key (SCHEMA$<PATH>$<ATTR>)
3. Fall back to schema-provided value (untranslated)

Co-authored-by: openhands <openhands@all-hands.dev>
…a fields

Add 154 SCHEMA translation keys covering:
- All fields from /agent-schema endpoint (general, llm, condenser, verification)
- All fields from /conversation-schema endpoint
- Section labels (General, LLM, Condenser, Verification)
- Choice labels for select fields (reasoning_effort, reasoning_summary, critic_mode, security_analyzer)

Keys follow the convention: SCHEMA$<PATH>$<ATTR>
Examples:
- SCHEMA$LLM$REASONING_EFFORT$LABEL
- SCHEMA$VERIFICATION$CRITIC_MODE$DESCRIPTION
- SCHEMA$LLM$REASONING_EFFORT$CHOICE$HIGH

English values are populated; other languages have empty strings
for future translation contributions.

Co-authored-by: openhands <openhands@all-hands.dev>
Add translations for the most commonly used schema fields including:
- Core LLM fields (model, API key, base URL, temperature, etc.)
- AWS credential fields
- OpenRouter fields
- Retry/timeout fields
- Condenser fields
- Verification fields
- Section labels
- Choice labels

Languages: English, Japanese, Chinese (Simplified/Traditional), Korean,
Norwegian, Italian, Portuguese, Spanish, Arabic, French, Turkish, German,
Ukrainian, Catalan

Some lesser-used fields retain English-only text for future translation
contributions.

Co-authored-by: openhands <openhands@all-hands.dev>
- Create common resolveSchemaFieldText() function used by:
  - resolveSchemaFieldLabel()
  - resolveSchemaFieldDescription()
  - resolveSchemaFieldSectionLabel()
- Add warning logging when falling back to untranslated schema values
  (logs to console.warn with key and fallback value)
- Update resolveSchemaChoiceLabel to also log warnings on fallback
- Add tests for warning logging behavior (5 new tests)

Note: i18next already logs missing keys in debug mode (development),
but this explicit warning helps identify untranslated schema fields
in production environments as well.

Co-authored-by: openhands <openhands@all-hands.dev>
Fill in translations for all 64 SCHEMA keys across all 15 supported
languages (ja, zh-CN, zh-TW, ko-KR, no, it, pt, es, ar, fr, tr, de,
uk, ca).

This includes:
- LLM configuration labels/descriptions (extra headers, force string
  serializer, input/output costs, token limits, temperature, top_k,
  top_p, streaming, seed, reasoning effort/summary, etc.)
- Verification/Critic settings (confirmation mode, critic enable,
  critic mode choices, thresholds, iterative refinement, security
  analyzer, etc.)

Co-authored-by: openhands <openhands@all-hands.dev>
@tofarr
Copy link
Copy Markdown
Collaborator

tofarr commented Apr 15, 2026

Internationalized keys now working:
image

tofarr and others added 4 commits April 15, 2026 14:16
Restore _create_condenser, _create_security_analyzer_from_string,
_select_confirmation_policy, and _set_security_analyzer_from_settings
methods that were accidentally removed during merge conflict resolution.

Also restore necessary imports: AgentType, LLMSummarizingCondenser, LLM,
and security-related classes.

Co-authored-by: openhands <openhands@all-hands.dev>
Copy link
Copy Markdown
Collaborator

@tofarr tofarr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's get this in!

@tofarr tofarr merged commit b4da0e1 into main Apr 15, 2026
28 checks passed
@tofarr tofarr deleted the openhands/issue-2228-gui-settings-schema branch April 15, 2026 23:00
@mamoodi mamoodi added the release:cloud-1.23.0 Cloud release 1.23.0 label Apr 22, 2026 — with OpenHands AI
devin-ai-integration Bot pushed a commit to OrpingtonClose/OpenHands that referenced this pull request Apr 26, 2026
Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: hieptl <hieptl.developer@gmail.com>
Co-authored-by: tofarr <tofarr@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release:cloud-1.23.0 Cloud release 1.23.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incorporate SDK-managed settings schema into the OpenHands GUI

6 participants