Skip to content

Fix and update the token limits handling#754

Merged
Mingholy merged 3 commits into
QwenLM:mainfrom
zhutao100:pr-tokenLimits
Oct 14, 2025
Merged

Fix and update the token limits handling#754
Mingholy merged 3 commits into
QwenLM:mainfrom
zhutao100:pr-tokenLimits

Conversation

@zhutao100

Copy link
Copy Markdown
Contributor

This PR enhances the token limits functionality by:

  1. Adding support for new models:

    • GLM-4.6 with 200k token limit
    • DeepSeek-v3.2-exp with 128k token limit
  2. Improving the model name normalization logic to:

    • Handle more date-like suffixes (e.g., deepseek-r1-0528)
    • Handle -exp suffixes in model names
    • Better distinguish between version numbers that should be stripped vs. kept
    • Improve handling of version numbers with and without the 'v' prefix

The changes are focused on improving the accuracy of token limit detection for various AI models, which is critical for ensuring proper handling of different model capabilities in the Qwen Code CLI.

Testing Matrix

🍏 🪟 🐧
npm run 👍
npx 👍
Docker
Podman - -
Seatbelt - -

@Mingholy Mingholy left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Mingholy Mingholy merged commit 9d66462 into QwenLM:main Oct 14, 2025
33 of 34 checks passed
reski-rukmantiyo pushed a commit to reski-rukmantiyo/qwen-code that referenced this pull request Oct 15, 2025
* fix: make token limits regex normalize e.g. `some-model-1.1` -> `some-model` while preserve e.g. `gpt-4.1` as-is.

* feat: update token limits regex for latest models `GLM-4.6`, `deepseek-v3.2-exp`.

* feat: add exact token limit number 202752 per the model config file for `GLM-4.6`.
@zhutao100 zhutao100 deleted the pr-tokenLimits branch November 1, 2025 05:57
andywinnock pushed a commit to andywinnock/qwen-code that referenced this pull request Nov 3, 2025
* fix: make token limits regex normalize e.g. `some-model-1.1` -> `some-model` while preserve e.g. `gpt-4.1` as-is.

* feat: update token limits regex for latest models `GLM-4.6`, `deepseek-v3.2-exp`.

* feat: add exact token limit number 202752 per the model config file for `GLM-4.6`.
xaelistic pushed a commit to xaelistic/qwen-code that referenced this pull request Jun 7, 2026
Address multiple possible memory leaks found bystatic analysis of the codebase. The primary source of the leaks was event listeners on child processes and global objects that were not being properly removed, potentially causing their closures to be retained in memory indefinitely particularly for processes that did not exit.

There are two commits. A larger one made by gemini CLI and a smaller one by me to make sure we always disconnect child processes as part of the cleanup methods. These changes may not actually fix any leaks but do look like reasonable defensive coding to avoid leaking event listeners or child processes.

The following files were fixed:
This is Gemini's somewhat overconfident description of what it did.

packages/core/src/tools/shell.ts: Fixed a leak where an abortSignal listener was not being reliably removed.
packages/cli/src/utils/readStdin.ts: Fixed a significant leak where listeners on process.stdin were never removed.
packages/cli/src/utils/sandbox.ts: Fixed leaks in the imageExists and pullImage helper functions where listeners on spawned child processes were not being removed.
packages/core/src/tools/grep.ts: Fixed three separate leaks in the isCommandAvailable check and the git grep and system grep strategies due to un-removed listeners on child processes.
packages/core/src/tools/tool-registry.ts: Corrected a leak in the execute method of the DiscoveredTool class where listeners on the spawned tool process were not being removed.
xaelistic pushed a commit to xaelistic/qwen-code that referenced this pull request Jun 7, 2026
* fix: make token limits regex normalize e.g. `some-model-1.1` -> `some-model` while preserve e.g. `gpt-4.1` as-is.

* feat: update token limits regex for latest models `GLM-4.6`, `deepseek-v3.2-exp`.

* feat: add exact token limit number 202752 per the model config file for `GLM-4.6`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants