Skip to content

fix(desktop): keep model runtime state per session#40163

Closed
pinguarmy wants to merge 5 commits into
NousResearch:mainfrom
pinguarmy:fix/desktop-model-runtime-ui
Closed

fix(desktop): keep model runtime state per session#40163
pinguarmy wants to merge 5 commits into
NousResearch:mainfrom
pinguarmy:fix/desktop-model-runtime-ui

Conversation

@pinguarmy

@pinguarmy pinguarmy commented Jun 5, 2026

Copy link
Copy Markdown

Summary

  • keep Desktop cached-session runtime state in sync with the active chat's model/provider/reasoning/fast/yolo fields
  • restore persisted model/provider/reasoning/service-tier runtime when resuming stored sessions instead of inheriting the latest global model from another chat
  • persist live model/reasoning/fast changes back to the session row so future resumes keep the chat's latest runtime
  • include provider in session.info and emit session.info after reasoning/fast/model changes
  • add shared formatting for the footer/model label and expose reasoning/fast state in the model menu

Root cause

Desktop had multiple stale-state paths:

  1. The renderer cached one set of footer atoms globally, so switching sessions could keep showing another chat's runtime state.
  2. session.resume rebuilt the backend agent from current global config rather than the stored session row. That meant an older chat whose DB row said gpt-5.4 + high reasoning could still resume as the latest global gpt-5.5 + medium reasoning, so the frontend received the wrong session.info payload even after renderer fixes.
  3. Live runtime changes made inside a session (model switch, reasoning effort, fast/service tier) were emitted to the current frontend but not consistently persisted for the next resume.

Test plan

  • Manual local verification: after restarting Hermes, switching between stored sessions changed the footer from GPT-5.5 · Med to the older chat's stored runtime as expected.
  • /Users/peterhao/.hermes/hermes-agent/.venv/bin/python -m pytest tests/test_tui_gateway_server.py::test_session_resume_passes_stored_runtime_to_agent tests/test_tui_gateway_server.py::test_persist_live_session_runtime_preserves_resume_metadata tests/test_tui_gateway_server.py::test_make_agent_uses_session_runtime_overrides tests/test_tui_gateway_server.py::test_session_resume_uses_parent_lineage_for_display tests/test_tui_gateway_server.py::test_make_agent_defaults_to_90 -q -o 'addopts='
  • /Users/peterhao/.hermes/hermes-agent/.venv/bin/python -m py_compile tui_gateway/server.py tests/test_tui_gateway_server.py
  • PATH=/Users/peterhao/.nvm/versions/node/v24.14.0/bin:$PATH npm run type-check --workspace apps/desktop
  • from apps/desktop: PATH=/Users/peterhao/.nvm/versions/node/v24.14.0/bin:$PATH ../../node_modules/.bin/vitest run src/app/session/hooks/use-model-controls.test.tsx src/lib/model-status-label.test.ts src/store/session.test.ts --environment jsdom

@pinguarmy

Copy link
Copy Markdown
Author

Follow-up fix pushed in c92f3a0:

  • prevents refreshCurrentModel() from overwriting the active session footer state with global/default model info
  • clears stale footer runtime badges when entering a fresh draft
  • reloads global draft defaults for model/reasoning/fast when no runtime session is active
  • adds a focused regression test for the "global refresh clobbers active session footer" case

This addresses the remaining report that the lower status bar model label could still stay wrong after switching sessions / returning to a new draft even after the earlier cached-session runtime-state fix.

@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/tui Terminal UI (ui-tui/ + tui_gateway/) labels Jun 5, 2026
@pinguarmy

Copy link
Copy Markdown
Author

Found the remaining bug and pushed a backend fix in 8c82bee2f.

The renderer fixes were necessary but not sufficient: session.resume rebuilt the backend agent from the current global config. So after using GPT-5.5/medium in one chat, resuming an older chat could still construct that runtime as GPT-5.5/medium even if the stored session row said GPT-5.4/high.

New behavior:

  • session.resume parses the stored session row (model, billing_provider, JSON model_config)
  • _make_agent accepts session-scoped runtime overrides
  • resumed session.info now reports the stored chat's model/provider/reasoning/service-tier instead of the latest global model

I verified against Peter's actual DB rows: 20260606_000556_ffaf51 resolves to gpt-5.4 + openai-codex + high reasoning, while the current chat resolves to gpt-5.5 + medium.

@pinguarmy

Copy link
Copy Markdown
Author

Final sweep found and fixed one more persistence edge case in e899f497a.

After the resume fix, the old-session footer state was restored correctly. The remaining weird case was future resumes after live runtime changes: a model switch / reasoning change / fast toggle could update the current frontend via session.info but not necessarily persist the latest runtime into the session row for the next resume.

New follow-up:

  • persists active session runtime after model switch, reasoning effort changes, and fast/service-tier toggles
  • stores provider/reasoning/service-tier in model_config while preserving existing metadata like _branched_from
  • makes stored model_config.provider take precedence over older billing-provider fallback when resuming

Manual verification from Peter: restarting Hermes and switching sessions now updates the footer correctly.

@pinguarmy pinguarmy force-pushed the fix/desktop-model-runtime-ui branch from e899f49 to c9de80f Compare June 7, 2026 14:07
@pinguarmy

Copy link
Copy Markdown
Author

Rebased this branch onto current origin/main (a317e5493) after upstream merged related Desktop runtime fixes (#41120 and #41121).

Current status:

Local verification:

  • python -m py_compile tui_gateway/server.py
  • backend focused runtime tests: 11 passed, 201 deselected
  • renderer focused tests on the combined rebased local stack: 15 passed
  • Desktop type-check and production build passed on the combined rebased local stack

@OutThisLife

Copy link
Copy Markdown
Collaborator

Review

Approach is sound and the diagnosis is correct, but the branch is stale/conflicting against current main again and needs a rebase before merge, plus a couple of correctness gaps. Net: changes requested.

What's good

  • Diagnosis is accurate — all three stale-state paths (global footer atoms, resume rebuilding from global config, live changes not persisting) are real, and the layered renderer + backend fix is the right shape.
  • Per-session footer state (types.ts, chat-runtime.ts, use-message-stream.ts, use-session-actions.ts, use-session-state-cache.ts) cleanly extends the existing ClientSessionState cache.
  • refreshCurrentModel() bail-when-$activeSessionId is the correct fix for the "global refresh clobbers active session" bug, with a focused regression test.
  • Tests assert relationships (resume → agent kwargs; persist preserves _branched_from) rather than snapshots — good per the repo's testing policy.
  • Backward compatible: the dict model_override path (custom-endpoint switches from fix(desktop): preserve configured base_url on same-provider model switch #41121) still hits isinstance(model_override, dict), so _reset_session_agent is unaffected.

Blocking — branch is stale again

merge-tree against current main shows real conflicts:

  1. _apply_model_switchmain now calls _restart_slash_worker(sid, session) (two args); this PR still calls _restart_slash_worker(session). Resolution must keep the two-arg form and add _persist_live_session_runtime(session).
  2. use-session-state-cache.ts (highest-risk)main added a deliberate reference-equality guard in flushPendingViewState (if (!sameMessageList(nextMessages, currentMessages)) setMessages(...)) to stop scroll jerk on session.info heartbeats. This PR's hunk reverts that to an unconditional setMessages(...). The new model/provider/reasoning setters must be added alongside the sameMessageList guard, not in place of it.
  3. _make_agent signature + model-menu-panel.tsx comment conflicts are mechanical.

Correctness concerns

  • Resume loses custom base_url/api_key/api_mode. _stored_session_runtime_overrides returns a scalar model_override + provider/reasoning/service_tier but never the endpoint credentials (_runtime_model_config doesn't persist them either). A session that switched to a custom endpoint (the fix(desktop): preserve configured base_url on same-provider model switch #41121 case) will, on resume, re-resolve the provider from config via resolve_runtime_provider and can land on the wrong base_url. At minimum document the gap; ideally persist base_url/api_mode into model_config.
  • Double DB write in _persist_live_session_runtime. It calls update_session_model(...) then update_session_meta(..., model=None) — two _execute_write transactions. Since update_session_meta already does model = COALESCE(?, model), collapse to a single update_session_meta(session_key, json.dumps(model_config), model).
  • Resumed session doesn't repopulate session["model_override"]. After a DB-restored resume, a later /new (_reset_session_agent) passes model_override=session.get("model_override") = None, reverting to global. Minor, but the resume-restore and /new-preserve paths then use two different sources of truth.

Minor

  • The model-menu-panel.tsx change (stop labeling inactive rows "Fast") is a reasonable UX fix but scope-adjacent to the runtime-persistence core.
  • No CI has run on the branch (no checks reported). After rebasing, run scripts/run_tests.sh tests/test_tui_gateway_server.py + the desktop vitest/type-check on the merged tree, not a local stack.

Suggested path

  1. Rebase onto current main; carefully resolve use-session-state-cache.ts and _restart_slash_worker (keep main's optimizations).
  2. Collapse the double DB write.
  3. Decide whether to persist endpoint fields on resume, or document the gap.
  4. Re-run backend + desktop tests on the rebased tree.

郝鹏宇 added 5 commits June 10, 2026 11:16
(cherry picked from commit f72ee87d99ee38cb7b5badeb9a8af869bb92073a)
(cherry picked from commit d91942ebd4671ff857b5c8526dbf133f04782ecb)
(cherry picked from commit 32b3793418257617b8da57e26151f079c2620d00)
(cherry picked from commit c58467779436dcef44a80ad55b52664752dc0837)
@pinguarmy pinguarmy force-pushed the fix/desktop-model-runtime-ui branch from c9de80f to 4924f7a Compare June 10, 2026 10:28
@pinguarmy

Copy link
Copy Markdown
Author

Rebased onto current upstream/main (a72bb0375) and pushed follow-up commit 4924f7af7.

Addressed the review points:

  • resolved _apply_model_switch conflict by keeping _restart_slash_worker(sid, session) and adding _persist_live_session_runtime(session) after the switch
  • resolved use-session-state-cache.ts by preserving the sameMessageList(...) guard and adding the per-session runtime setters alongside it, not replacing it
  • collapsed live runtime persistence to a single update_session_meta(..., model) DB write, with update_session_model only as a legacy fallback when update_session_meta is unavailable
  • persist/restore custom endpoint base_url and api_mode in model_config, while deliberately not storing raw api_key in the session DB
  • repopulate session["model_override"] after DB-restored resume so later /new rebuilds use the same restored runtime source of truth

Verification on the rebased branch:

  • python -m py_compile tui_gateway/server.py tests/test_tui_gateway_server.py passed
  • focused backend runtime tests: 5 passed
  • npm run type-check --workspace apps/desktop passed
  • focused desktop vitest: 24 passed
  • scripts/run_tests.sh tests/test_tui_gateway_server.py: 255 passed, 1 failed; the failing test_browser_manage_connect_default_local_reports_launch_hint also fails unchanged on clean upstream/main (a72bb0375) in this local macOS environment, so I’m treating it as a pre-existing/local-env failure rather than introduced by this branch.

I also ran an independent review pass over the rebased branch; no blockers were found.

@OutThisLife

Copy link
Copy Markdown
Collaborator

Superseded by #43702.

Salvaged your five commits (authorship preserved via cherry-pick) onto current main and opened #43702. All of the review feedback in this thread was already handled by your latest revision — verified on the rebased tree:

  • two-arg _restart_slash_worker(sid, session) + _persist_live_session_runtime(session) in _apply_model_switch
  • per-session runtime setters added alongside the sameMessageList(...) guard in use-session-state-cache.ts (not replacing it)
  • single update_session_meta(..., model) DB write, update_session_model only as legacy fallback
  • custom endpoint base_url/api_mode persisted/restored in model_config
  • session["model_override"] repopulated after a DB-restored resume

Verification on the merged tree: tests/test_tui_gateway_server.py 255 passed (the lone test_browser_manage_connect_default_local_reports_launch_hint failure reproduces identically on clean main in this local macOS env and is unrelated to this diff), desktop typecheck clean, focused desktop vitest 24 passed.

Closing in favor of #43702. Thanks @pinguarmy!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/tui Terminal UI (ui-tui/ + tui_gateway/) P3 Low — cosmetic, nice to have type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants