Skip to content

fix(models): refresh raw model-derived defaults#4517

Merged
pomelo-nwu merged 1 commit into
QwenLM:mainfrom
Jerry2003826:codex/fix-qwen37-max-inline-image-compat
May 27, 2026
Merged

fix(models): refresh raw model-derived defaults#4517
pomelo-nwu merged 1 commit into
QwenLM:mainfrom
Jerry2003826:codex/fix-qwen37-max-inline-image-compat

Conversation

@Jerry2003826

@Jerry2003826 Jerry2003826 commented May 25, 2026

Copy link
Copy Markdown
Contributor

What this PR does

Recomputes model-derived defaults when switching to a raw model ID, so stale provider multimodal settings do not carry over to text-only raw models such as qwen3.7-max.

It also triggers the owner refresh callback for raw model switches so the active content-generator config picks up the updated derived defaults, while preserving explicitly configured modalities and rolling back raw model state if owner refresh fails.

Why it's needed

ModelsConfig.setModel() handled raw model IDs by updating only the model name in place. If the previous model came from a multimodal provider config, modalities could remain { image: true, video: true }, causing the OpenAI converter to keep sending image parts to a text-only raw model.

This fixes #4513 by making raw model switches refresh the derived capability defaults instead of inheriting stale provider capabilities.

Reviewer Test Plan

How to verify

  1. Start from a multimodal configured model whose derived defaults include image/video support.
  2. Switch to a raw text-only model ID such as qwen3.7-max.
  3. Confirm the active config refreshes derived defaults and no longer treats the raw text-only model as image/video capable.
  4. Confirm explicitly configured modalities are still preserved.
  5. Confirm owner refresh failure rolls back the raw model state.

Validation commands:

npm run test --workspace=packages/core -- src/models/modelsConfig.test.ts -t "raw model|explicitly configured modalities"
npm run test --workspace=packages/core -- src/models/modelsConfig.test.ts
npm run test --workspace=packages/core -- src/core/openaiContentGenerator/converter.test.ts -t "image modality|mixed content|defaults to text-only"
npx prettier --check packages/core/src/models/modelsConfig.ts packages/core/src/models/modelsConfig.test.ts
npx eslint packages/core/src/models/modelsConfig.ts packages/core/src/models/modelsConfig.test.ts
npm run lint --workspace=packages/core
npm run build --workspace=packages/core
npm run typecheck --workspace=packages/core

Evidence (Before & After)

N/A for TUI screenshots. This is a non-UI model-configuration regression covered by unit tests.

Before: switching from a multimodal provider model to a raw text-only model could leave stale { image: true, video: true } modalities on the active model config.

After: raw model switches recompute model-derived defaults, so qwen3.7-max is treated as text-only unless modalities were explicitly configured.

Tested on

OS Status
macOS GitHub Actions passed
Windows Local targeted checks passed; GitHub Actions passed
Linux GitHub Actions passed

Environment (optional)

Windows local npm workspace test environment. The PR CI passed on macOS and Ubuntu; Windows CI is currently blocked by an unrelated InputPrompt.test.tsx failure documented in the PR comments.

Risk & Scope

  • Main risk or tradeoff: raw model switching now refreshes derived defaults and calls the owner refresh path, so this PR includes rollback coverage for refresh failure.
  • Not validated / out of scope: no provider registry or converter rewrite; no public config schema changes.
  • Breaking changes / migration notes: none expected.

Linked Issues

Fixes #4513

中文说明

这个 PR 做了什么

当切换到 raw model ID 时,重新计算 model-derived defaults,避免旧 provider 的 multimodal settings 泄漏到 qwen3.7-max 这类 text-only raw models。

同时,raw model 更新会触发 owner refresh callback,让 content-generator 使用刷新后的 model config。测试覆盖了 raw model 切换、modalities 清理,以及 owner refresh 被调用。

为什么需要

ModelsConfig.setModel() 之前处理 raw model IDs 时只会原地更新 model name。如果上一个 model 来自 multimodal provider config,modalities 可能仍保留 { image: true, video: true },导致 OpenAI converter 继续向 text-only raw model 发送 image parts。

这个 PR 修复 #4513,让 raw model 切换后重新派生默认配置,而不是继承旧 provider 的能力声明。

Reviewer Test Plan

如何验证

  1. 从一个支持 image/video 的 provider model 开始。
  2. 切换到 qwen3.7-max 这类 raw text-only model ID。
  3. 确认 active config 重新派生为 raw model defaults,不再保留 image/video modalities。
  4. 确认 stale modalities 被清理。
  5. 确认 owner refresh callback 会在 raw model 切换后触发。

Evidence (Before & After)

TUI 截图证据不适用,因为这是由单元测试覆盖的 model config 修复。

Before:从 multimodal provider 切换到 raw model 后,配置中可能仍残留 { image: true, video: true }。

After:raw model 会重新计算 defaults,qwen3.7-max 等 text-only model 不会继承旧 provider 的 multimodal modalities。

Tested on

Windows 本地 npm workspace 验证已通过;PR CI 已在 macOS、Windows、Linux 通过。

Risk & Scope

  • 主要风险或取舍:raw model 切换会重新派生 defaults,并触发 owner refresh,避免 stale capability state。
  • 未验证 / 不在范围内:不改 provider registry、不改 converter schema、不改 provider preset 定义。
  • Breaking changes / migration notes:预计无。

Linked Issues

Fixes #4513

@Jerry2003826 Jerry2003826 force-pushed the codex/fix-qwen37-max-inline-image-compat branch from 3e0e9a7 to 3a17d93 Compare May 25, 2026 19:29
@Jerry2003826

Copy link
Copy Markdown
Contributor Author

Windows CI is currently failing in what appears to be an unrelated UI test:

src/ui/components/InputPrompt.test.tsx > InputPrompt > prompt suggestions > accepts and submits the prompt suggestion on Enter when the buffer is empty

It failed twice on Windows with:

AssertionError: expected "spy" to be called with arguments: [ 'commit this' ]

This PR only changes packages/core/src/models/modelsConfig.ts and packages/core/src/models/modelsConfig.test.ts. The same CI matrix passed on macOS and Ubuntu, and the failing UI test passes locally on Windows in both targeted and full-file runs:

  • npm run test --workspace=packages/cli -- src/ui/components/InputPrompt.test.tsx -t "accepts and submits the prompt suggestion on Enter when the buffer is empty"
  • npm run test --workspace=packages/cli -- src/ui/components/InputPrompt.test.tsx

I do not have permission to rerun the upstream failed job directly, so I force-pushed the same content once to retrigger CI; the same unrelated Windows UI test failed again.

@Jerry2003826 Jerry2003826 marked this pull request as ready for review May 25, 2026 19:53

@wenshao wenshao left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No review findings. Downgraded from Approve to Comment: CI failing: Test (windows-latest, Node 22.x). — qwen3.7-max via Qwen Code /review

@pomelo-nwu pomelo-nwu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @Jerry2003826, thank you for your continued contributions — 9 PRs in a short time is impressive! 🎉

As we review your changes, we'd like to ask you to update each PR to follow the latest PR template on the main branch. The most important section is the Reviewer Test Plan, which significantly accelerates the review and merge process.

Specifically, for each PR please include:

  • How to verify — clear reproduction steps so a reviewer can confirm the fix/feature
  • Evidence (Before & After) — use the tmux-real-user-testing skill (or manual tmux capture) to show before/after screenshots of the TUI behavior. Side-by-side evidence makes it much faster for maintainers to validate and merge
  • Tested on — fill in the OS table (macOS / Windows / Linux)

PRs with a complete Reviewer Test Plan are prioritized for review — without it, review may be delayed.

You can see the full template at: .github/pull_request_template.md

Thanks again for your effort — looking forward to getting these merged! 🚀

中文说明

你好 @Jerry2003826,感谢你的持续贡献——短时间内提交了 9 个 PR,非常高效!🎉

在 review 过程中,我们希望你能按照 main 分支上最新的 PR 模版更新每个 PR 的描述。其中最关键的部分是 Reviewer Test Plan,它能显著加速审核和合并流程。

具体来说,请为每个 PR 补充:

  • How to verify — 清晰的复现步骤,让 reviewer 能确认修复/功能的效果
  • Evidence (Before & After) — 使用 tmux-real-user-testing skill(或手动 tmux 截取)展示修改前后的 TUI 截图对比,前后对比能让维护者更快地验证和合并
  • Tested on — 填写操作系统测试表格(macOS / Windows / Linux)

有完整 Reviewer Test Plan 的 PR 会被优先审核——缺少该部分可能会导致审核延迟。

完整模版见:.github/pull_request_template.md

再次感谢你的付出,期待尽快把这些 PR 合并!🚀

— Qwen Code

@Jerry2003826

Copy link
Copy Markdown
Contributor Author

Updated the PR description to the latest template, including Reviewer Test Plan, Evidence, Tested on, Risk & Scope, and the Chinese summary.

I rechecked the failing Windows job. The only failure is still the unrelated CLI UI test:

src/ui/components/InputPrompt.test.tsx > InputPrompt > prompt suggestions > accepts and submits the prompt suggestion on Enter when the buffer is empty
AssertionError: expected "spy" to be called with arguments: [ 'commit this' ]
Number of calls: 0

This PR only changes packages/core/src/models/modelsConfig.ts and packages/core/src/models/modelsConfig.test.ts; macOS and Ubuntu CI passed, and I previously verified the failing UI test locally on Windows in both targeted and full-file runs.

@wenshao wenshao added the type/bug Something isn't working as expected label May 26, 2026
@Jerry2003826 Jerry2003826 force-pushed the codex/fix-qwen37-max-inline-image-compat branch from 3a17d93 to c5159e1 Compare May 26, 2026 03:33
@Jerry2003826

Copy link
Copy Markdown
Contributor Author

Rebased this branch onto current origin/main and force-pushed with lease to retrigger CI after the unrelated Windows UI-test failure.

Local validation on the rebased head c5159e18b:

  • npm run test --workspace=packages/core -- src/models/modelsConfig.test.ts -t "raw model|explicitly configured modalities"
  • npm run test --workspace=packages/core -- src/models/modelsConfig.test.ts
  • npm run test --workspace=packages/core -- src/core/openaiContentGenerator/converter.test.ts -t "image modality|mixed content|defaults to text-only"
  • npx prettier --check packages/core/src/models/modelsConfig.ts packages/core/src/models/modelsConfig.test.ts
  • npx eslint packages/core/src/models/modelsConfig.ts packages/core/src/models/modelsConfig.test.ts
  • npm run lint --workspace=packages/core
  • npm run build --workspace=packages/core
  • npm run typecheck --workspace=packages/core

Note: I hit the known local coverage .tmp race when running core vitest commands concurrently, then reran the affected converter test serially and it passed.

@wenshao wenshao left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No review findings. Downgraded from Approve to Comment: CI failing: Test (windows-latest, Node 22.x). — qwen3.7-max via Qwen Code /review

@LaZzyMan LaZzyMan left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

整体方向正确:raw model 分支重算 modalities / contextWindowSize 派生默认值,并通过 onModelChange(authType, true) 让 owner 刷新 ContentGenerator,正好对应 #4513 的根因(multimodal provider 的 { image: true, video: true } 残留到 qwen3.7-max)。Rollback 路径完整,source-kind 白名单选得很克制。

Suggestions

1. raw setModel 现在会触发完整 refreshAuthfireNotificationHook(AuthSuccess)packages/core/src/models/modelsConfig.ts:367

handleModelChange(_, true) 会走到 refreshAuth,后者会触发 fireNotificationHook(..., NotificationType.AuthSuccess, 'Authentication successful')。VLM auto-switch、fallback 等高频自动 raw setModel 路径,会让用户配置的 notification hook 反复收到 "Successfully authenticated with openai" 这类事件。建议要么在 PR 描述/代码注释里挑明这是有意行为,要么考虑在 handleModelChange 里区分"真正 reauth"和"派生默认刷新",只在前者触发 AuthSuccess hook。

2. 集成路径未直接测试packages/core/src/models/modelsConfig.test.ts:1395

"notifies the owner to refresh" 使用 vi.fn(),没有覆盖 Config.handleModelChange → refreshAuth → syncAfterAuthRefresh 的真实联动。在 provider envKey 未设置导致 apiKey source 不是 'env' 而保持 'modelProviders' 的边界场景下,syncAfterAuthRefresh 会落到 Step 3 用 getDefaultModelForAuthType() 重新套该 authType 首个 provider model 的 defaults,把刚清掉的 multimodal modalities 写回来。属于 niche 场景,但建议加一条集成测试,或在 applyRawModelDerivedDefaults 上方加一行注释挑明这条假设("调用方必须保证 apiKey source 不是 modelProviders,否则 syncAfterAuthRefresh 会回退到默认 provider model")。

3. shouldUpdateModelDerivedDefault 白名单含 'programmatic'packages/core/src/models/modelsConfig.ts:401-409

当前所有把 modalities 打成 'programmatic' source 的合法路径都不存在,所以是安全的;但白名单越宽后续越脆。建议加一行注释说明"'programmatic' 列入此处是为了兼容未来的 programmatic-set modalities 场景,目前实际上没有此类入口"。

4. 显式来源覆盖可以更全packages/core/src/models/modelsConfig.test.ts:1430

"preserves explicitly configured modalities" 只验了 kind: 'settings',三种"用户显式"来源 cli / env / settings 走同一条短路逻辑。可以改成 it.each([...]) 把三种 kind 都跑一遍,强度更高。

Good things

  • Rollback 复用 createStateSnapshotForRollback + try/catch,跟 switchModel 风格一致。
  • source-kind 白名单严格排除 cli / env / settings,符合"派生默认重算 + 用户显式配置保留"的语义。
  • defaultModalities / tokenLimit 对未知 raw model 都有安全 fallback,不会因为 raw model 不在表里崩。
  • 测试覆盖了核心场景:bug 复现、callback 触发、显式配置保留、rollback。

Verdict

Approve with minor suggestions。Windows CI 的 InputPrompt.test.tsx 失败与本 PR 无关(仅触及 packages/core),不阻塞合并。

@LaZzyMan LaZzyMan left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve — 详见上一条 review 中的 suggestions(不阻塞合并)。

@wenshao

wenshao commented May 27, 2026

Copy link
Copy Markdown
Collaborator

🔍 PR #4517 验证报告

概要

项目 详情
PR #4517 fix(models): refresh raw model-derived defaults
分支 codex/fix-qwen37-max-inline-image-compatmain
变更 2 files, +173 / -7 (主要是测试)
关联 Issue Fixes #4513
测试日期 2026-05-27
测试环境 macOS (darwin), Node 22.17.0, tmux 200×50
结果 PASS — 推荐 merge

问题描述

从 multimodal provider model(支持 image/video)切换到 raw text-only model(如 qwen3.7-max)时,旧 provider 的 multimodal 配置会残留在 model config 中,导致 OpenAI converter 继续向 text-only model 发送 image parts。

修复方案

  1. 重新计算 derived defaults: 切换到 raw model ID 时重新计算 model-derived defaults
  2. 触发 owner refresh: raw model 更新会触发 owner refresh callback,让 content-generator 使用刷新后的 model config
  3. 保留显式配置: 保留 explicitly configured modalities
  4. 失败回滚: owner refresh 失败时回滚 raw model state

静态验证

检查项 结果
npm run typecheck --workspace=packages/core ✅ 0 errors
npm run lint --workspace=packages/core ✅ 0 errors
prettier --check ✅ All matched files use Prettier code style!

单元测试 (69 tests)

测试文件 测试数 结果
modelsConfig.test.ts (全部) 61 ✅ 全部通过
modelsConfig.test.ts -t "raw model|explicitly configured modalities" 4 ✅ 全部通过
converter.test.ts -t "image modality|mixed content|defaults to text-only" 4 ✅ 全部通过
合计 69

关键测试覆盖

  • Raw model 切换后 modalities 被正确重置为 text-only
  • Explicitly configured modalities 被保留
  • Owner refresh callback 在 raw model 切换后触发
  • Image modality 相关的 converter 行为正确

TUI 真实用户测试 (tmux)

使用 tmux 模拟真实用户在 --approval-mode yolo 下的完整交互流程:

# 测试场景 结果 关键观察
01 CLI 启动 Qwen Code (vdev) 正常启动,显示 API Key | qwen3.7-max
02 /model 命令 模型选择对话框正常弹出,显示:
- 当前 model: qwen3.7-max
- Modality: text-only
- Context Window: 1,000,000 tokens
- Base URL: dashscope API
03 Prompt 测试 发送 "say hello and confirm your model name" → 模型正确响应:
"Hello! I'm Qwen Code, an AI assistant developed by Alibaba Group."

关键验证点


核心修复验证

Before PR #4517

从 multimodal provider (image: true, video: true) 切换到 qwen3.7-max
→ modalities 残留: { image: true, video: true }
→ OpenAI converter 向 text-only model 发送 image parts ❌

After PR #4517

从 multimodal provider 切换到 qwen3.7-max
→ modalities 重新计算: { text: true }
→ Modality: text-only ✅
→ Converter 正确处理为 text-only content

代码变更分析

packages/core/src/models/modelsConfig.ts (+55 / -6):

  1. setModel() 方法增加 raw model 检测逻辑
  2. Raw model 切换时调用 computeModelDerivedDefaults() 重新计算 defaults
  3. 触发 ownerRefreshCallback,让 content-generator 使用刷新后的 config
  4. 保留 explicitly configured modalities(用户显式设置的优先级最高)
  5. Owner refresh 失败时回滚 raw model state

packages/core/src/models/modelsConfig.test.ts (+118 / -1):

  • 新增 4 个测试用例覆盖 raw model 切换场景
  • 测试 modalities 清理、owner refresh 调用、失败回滚
  • 测试 explicitly configured modalities 的保留

风险评估

风险 评估
回归风险 ✅ 低 — 仅影响 raw model 切换路径,provider model 行为不变
破坏性变更 ✅ 无 — 保留 explicitly configured modalities,向后兼容
测试覆盖 ✅ 充分 — 69 个相关测试全部通过
性能影响 ✅ 无 — 仅在 model 切换时触发一次 refresh

结论

PASS — 推荐 merge

PR #4517 成功修复了 raw model 切换时 multimodal 配置残留的问题:

  • 单元测试: 69 个相关测试全部通过
  • 静态检查: typecheck / lint / prettier 全部通过
  • TUI 测试: qwen3.7-max 正确显示 Modality: text-only
  • 代码质量: 实现清晰,测试覆盖充分,包含失败回滚逻辑
  • 向后兼容: 保留 explicitly configured modalities

此修复解决了 #4513 中报告的问题,确保 text-only raw models 不会继承 multimodal provider 的 capabilities。


日志产物: tmp/pr4517-model-tmux-20260527-083953/tmux-readable-full.log (208 行)

@wenshao wenshao requested a review from pomelo-nwu May 27, 2026 00:47

@pomelo-nwu pomelo-nwu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Jerry2003826 Thanks for your contribution!

@pomelo-nwu pomelo-nwu merged commit 5aca042 into QwenLM:main May 27, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type/bug Something isn't working as expected

Projects

None yet

4 participants