Skip to content

feat(desktop): full tool-backend config (pickers + per-backend settings) in Settings#41232

Merged
teknium1 merged 2 commits into
mainfrom
feat/desktop-tts-stt-terminal-backend-dropdowns
Jun 7, 2026
Merged

feat(desktop): full tool-backend config (pickers + per-backend settings) in Settings#41232
teknium1 merged 2 commits into
mainfrom
feat/desktop-tts-stt-terminal-backend-dropdowns

Conversation

@teknium1

@teknium1 teknium1 commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

Summary

Every native tool backend the agent supports is now fully configurable from the desktop Settings UI — you can both PICK the backend from a dropdown AND tune that backend's own settings, with no config.yaml hand-editing. Originating request: Grok/xAI TTS, which was a fully-wired built-in backend with zero GUI surface.

Two layers were missing:

  1. Provider pickers — three backend-selector fields rendered as free-text boxes because their dotpath wasn't in ENUM_OPTIONS (helpers.ts::enumOptionsFor → free-text <Input> when absent).
  2. Per-backend config fields — most backends had no Settings fields for their voice/model/language/device, so picking them gave only defaults.

Also fixes a real gap: the STT provider dropdown was missing xai (Grok STT), which the transcription dispatcher handles but the config.py comment had omitted — the dispatch ladder (tools/transcription_tools.py) is the source of truth, not the comment.

Changes

  • constants.ts:
    • ENUM_OPTIONS pickers: tts.provider (10), stt.provider (6, +xai), terminal.backend (6) + new select sub-fields (stt.openai.model, stt.mistral.model, tts.openai.model, tts.elevenlabs.model_id, tts.neutts.device)
    • SECTIONS Voice: added xai/minimax/mistral/gemini/neutts/kittentts/piper TTS fields + openai/groq/mistral STT model fields
    • SECTIONS Advanced: terminal.docker_image / singularity_image / modal_image / daytona_image
    • FIELD_LABELS + FIELD_DESCRIPTIONS for every new field
  • i18n/{ja,zh,zh-hant}.ts: native translations for all new labels (English fallback already covers any gap via config-settings.tsx)
  • helpers.test.ts: dropdown-resolution tests for the pickers + sub-fields (18 tests)

Each option list mirrors the backend generator's accepted values (tools/tts_tool.py, tools/transcription_tools.py, hermes_cli/config.py). Secrets (provider API keys, modal/daytona tokens, ssh host/key) intentionally stay in Settings → Keys as env vars, not duplicated as config fields.

Validation

Before After
Backend pickers free-text boxes dropdowns (TTS 10 / STT 6 / terminal 6)
Grok STT in picker missing present
Per-backend voice/model/device config.yaml only Settings fields
Terminal images config.yaml only Settings fields

tsc --noEmit clean · 39/39 vitest (settings + i18n) · eslint clean on changed lines (one pre-existing import-order error at constants.ts:17, untouched, not CI-gated for desktop).

Infographic

Complete tool-backend config in desktop settings

@github-actions

github-actions Bot commented Jun 7, 2026

Copy link
Copy Markdown
Contributor

🔎 Lint report: feat/desktop-tts-stt-terminal-backend-dropdowns vs origin/main

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 10007 on HEAD, 10007 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 5195 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/tools Tool registry, model_tools, toolsets labels Jun 7, 2026
@teknium1 teknium1 changed the title feat(desktop): surface TTS/STT/terminal backends as Settings dropdowns feat(desktop): full tool-backend config (pickers + per-backend settings) in Settings Jun 7, 2026
teknium1 added 2 commits June 7, 2026 09:09
Every native tool backend that the agent supports now shows up as a
clickable picker in the desktop Settings UI instead of a free-text box.

Desktop Settings renders a config field as a <Select> only if its dotpath
is a key in ENUM_OPTIONS (helpers.ts::enumOptionsFor returns undefined ->
free-text <Input> otherwise). Three backend-selector fields were surfaced
in their sections but missing from the map, so users had to hand-type the
provider name and could reasonably assume it was unsupported:

- tts.provider — now lists all built-in TTS backends incl. xai (Grok)
- stt.provider — local/groq/openai/mistral/elevenlabs
- terminal.backend — local/docker/singularity/modal/daytona/ssh

Each list is kept in sync with its backend source of truth (TTS:
agent/tts_registry.py::_BUILTIN_NAMES + tools/tts_tool.py; STT + terminal:
hermes_cli/config.py / tools/terminal_tool.py). The existing
enumOptionsFor current-value-append keeps any hand-typed/legacy value
selected, and command-type TTS providers still work.

Reported for Grok/xAI TTS, which was already a fully-wired built-in
provider (tts.provider: xai + XAI_API_KEY) with no picker entry.
…ettings

Completes the backend-coverage pass: not just the provider PICKER but every
backend's own config fields are now tunable from desktop Settings, so a user
who picks (e.g.) Grok TTS can also set its voice/language without hand-editing
config.yaml.

Also fixes the STT provider dropdown: added 'xai' (Grok STT), which the
transcription dispatcher (tools/transcription_tools.py) handles but the
config.py comment had omitted — the dispatch ladder is the source of truth.

New Settings fields (Voice section):
- TTS xai (voice_id, language), minimax (model, voice_id), mistral
  (model, voice_id), gemini (model, voice), neutts (model, device),
  kittentts (model, voice), piper (voice)
- STT openai (model), groq (model), mistral (model)

New Settings fields (Advanced section):
- terminal docker_image / singularity_image / modal_image / daytona_image

New ENUM_OPTIONS dropdowns: stt.provider (+xai), stt.openai.model,
stt.mistral.model, tts.openai.model, tts.elevenlabs.model_id,
tts.neutts.device. Each list mirrors the backend generator's accepted values
(tools/tts_tool.py, tools/transcription_tools.py, hermes_cli/config.py).

i18n: FIELD_LABELS/FIELD_DESCRIPTIONS cover all locales via the English
fallback in config-settings.tsx; added native translations to ja/zh/zh-hant.

Secrets (provider API keys, modal/daytona tokens, ssh host/key) intentionally
stay in Settings -> Keys as env vars, not duplicated as config fields.
@teknium1 teknium1 force-pushed the feat/desktop-tts-stt-terminal-backend-dropdowns branch from a56786e to 891d45a Compare June 7, 2026 16:10
@teknium1 teknium1 merged commit 20fd0bd into main Jun 7, 2026
20 checks passed
@teknium1 teknium1 deleted the feat/desktop-tts-stt-terminal-backend-dropdowns branch June 7, 2026 17:05
agogo233 added a commit to agogo233/hermes-agent that referenced this pull request Jun 8, 2026
* upstream/main: (430 commits)
  fix(yuanbao): bound ws.close() so an idle server can't stall shutdown ~5s (NousResearch#40607)
  docs: add Urdu translation of README (NousResearch#40578)
  fix(hindsight): send only new-turn delta on append retains instead of whole session (NousResearch#40605)
  feat(gateway): render terminal tool calls as native bash code blocks on markdown platforms (NousResearch#41215)
  feat(desktop): stop the chat viewport from following streaming output (NousResearch#41414)
  chore(release): map AlchemistChaos co-author email for NousResearch#40135 salvage
  fix(desktop): recover chat after sleep/wake by revalidating a stale remote backend
  fix(web): make _has_env config-aware so SEARXNG_URL auto-detect honors Hermes config
  fix(web): honor Hermes config-aware SEARXNG_URL lookup
  install.sh: hint at root-owned npm cache when desktop npm install fails (NousResearch#39688)
  fix(tools): percent-encode non-ascii URL components
  fix(skills): browse shows full catalog, not first 5000 (NousResearch#41413)
  feat(desktop+gateway): remote media relay — attach images/PDFs and display gateway images over the network
  feat(desktop): full tool-backend config (pickers + per-backend settings) in Settings (NousResearch#41232)
  hardening(api-server): scan cron prompts on REST create/update for parity with the agent tool
  fix: skip MCP preflight content-type probe on reconnect when already ready (NousResearch#40604)
  fix(kanban): sweep deferred scratch parent on non-scratch child completion + tests
  fix: defer scratch workspace cleanup when task has active children (NousResearch#33774)
  feat(onboarding): opt-in structured profile-build path on first contact (NousResearch#41114)
  feat(compression): temporal anchoring in compaction summaries (NousResearch#41102)
  test(discord): align clarify/model-picker tests with fail-closed component auth (NousResearch#41338)
  chore(release): map Dusk1e and LaPhilosophie for approval fail-closed salvage (NousResearch#33844, NousResearch#33866, NousResearch#30964)
  fix(discord): fail closed for component button auth when no allowlist set
  fix(feishu): fail closed for update prompt card actions
  fix(slack): re-check gateway auth on approval and slash-confirm buttons
  fix: guard int(os.getenv()) casts against malformed env vars (NousResearch#40598)
  fix: respect Honcho env var fallback in doctor and honcho status
  chore(release): add synapsesx to AUTHOR_MAP for NousResearch#40495 salvage
  fix(research): keep tool_call/tool_response pairs intact when compressing trajectories
  fix(simplex): accept display name in SIMPLEX_ALLOWED_USERS
  fix(desktop): make the running-turn timer per-session (NousResearch#41182)
  test(approval): regression for shell-escape denylist bypass (NousResearch#36846, NousResearch#36847)
  fix(security): strip shell escapes in denylist normalizer; fail-closed on missing approval module
  fix(stream+output-cap): guard empty streams and parse OpenRouter output-cap errors (NousResearch#40589)
  fix(desktop): bootstrap falls back to installed agent install.sh on GitHub 404
  feat(dashboard): change UI font from the theme picker, independent of theme (NousResearch#41145)
  fix(cli): return bool (not None) when a destructive-slash confirmation is cancelled (NousResearch#40583)
  fix(desktop): preserve configured base_url on same-provider model switch (NousResearch#41121)
  fix(desktop): stop bare-URL autolinker swallowing trailing emphasis asterisks (NousResearch#41093)
  fix(cron): bound the desktop run-history query to one job (NousResearch#41088)
  fix(desktop): scope in-session /model switch per-session, stop process-env leak (NousResearch#41120)
  chore: map bmoore210 author email for PR NousResearch#40550 salvage
  fix(desktop): scope session list to active profile + longer timeout
  fix: harden gateway startup and turn persistence
  fix(computer_use): honor custom vision routing
  fix(aux): honor model.default_headers on auxiliary client too (NousResearch#40033)
  fix(agent): honor model.default_headers for custom OpenAI-compatible providers (NousResearch#40033)
  docs(i18n): port deep-audit corrections to zh-Hans mirror (NousResearch#41104)
  fix(compression): don't overwrite the -1 post-compression sentinel in preflight seed (NousResearch#36718)
  chore(release): map singhsanidhya741@gmail.com to sanidhyasin (NousResearch#41094)
  ...
changman pushed a commit to changman/hermes-agent that referenced this pull request Jun 10, 2026
…gs) in Settings (NousResearch#41232)

* feat(desktop): surface TTS/STT/terminal backends as Settings dropdowns

Every native tool backend that the agent supports now shows up as a
clickable picker in the desktop Settings UI instead of a free-text box.

Desktop Settings renders a config field as a <Select> only if its dotpath
is a key in ENUM_OPTIONS (helpers.ts::enumOptionsFor returns undefined ->
free-text <Input> otherwise). Three backend-selector fields were surfaced
in their sections but missing from the map, so users had to hand-type the
provider name and could reasonably assume it was unsupported:

- tts.provider — now lists all built-in TTS backends incl. xai (Grok)
- stt.provider — local/groq/openai/mistral/elevenlabs
- terminal.backend — local/docker/singularity/modal/daytona/ssh

Each list is kept in sync with its backend source of truth (TTS:
agent/tts_registry.py::_BUILTIN_NAMES + tools/tts_tool.py; STT + terminal:
hermes_cli/config.py / tools/terminal_tool.py). The existing
enumOptionsFor current-value-append keeps any hand-typed/legacy value
selected, and command-type TTS providers still work.

Reported for Grok/xAI TTS, which was already a fully-wired built-in
provider (tts.provider: xai + XAI_API_KEY) with no picker entry.

* feat(desktop): expose per-backend TTS/STT/terminal config fields in Settings

Completes the backend-coverage pass: not just the provider PICKER but every
backend's own config fields are now tunable from desktop Settings, so a user
who picks (e.g.) Grok TTS can also set its voice/language without hand-editing
config.yaml.

Also fixes the STT provider dropdown: added 'xai' (Grok STT), which the
transcription dispatcher (tools/transcription_tools.py) handles but the
config.py comment had omitted — the dispatch ladder is the source of truth.

New Settings fields (Voice section):
- TTS xai (voice_id, language), minimax (model, voice_id), mistral
  (model, voice_id), gemini (model, voice), neutts (model, device),
  kittentts (model, voice), piper (voice)
- STT openai (model), groq (model), mistral (model)

New Settings fields (Advanced section):
- terminal docker_image / singularity_image / modal_image / daytona_image

New ENUM_OPTIONS dropdowns: stt.provider (+xai), stt.openai.model,
stt.mistral.model, tts.openai.model, tts.elevenlabs.model_id,
tts.neutts.device. Each list mirrors the backend generator's accepted values
(tools/tts_tool.py, tools/transcription_tools.py, hermes_cli/config.py).

i18n: FIELD_LABELS/FIELD_DESCRIPTIONS cover all locales via the English
fallback in config-settings.tsx; added native translations to ja/zh/zh-hant.

Secrets (provider API keys, modal/daytona tokens, ssh host/key) intentionally
stay in Settings -> Keys as env vars, not duplicated as config fields.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/tools Tool registry, model_tools, toolsets P3 Low — cosmetic, nice to have type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants