bugfix for weight loading for qwen3-next#21313
bugfix for weight loading for qwen3-next#21313sglang-npu-bot merged 1 commit intosgl-project:mainfrom
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request addresses a bug in the weight loading mechanism for the Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request fixes a weight loading issue in the Qwen3-next model by updating the attribute name for the weight loader from weight_loader to _weight_loader. While this fixes the bug, I've added a comment regarding the practice of accessing private attributes directly, as it can lead to maintainability issues. Using a public API, if available, would be a more robust solution.
| self.in_proj_qkvz.weight._weight_loader = self._make_packed_weight_loader( | ||
| self.in_proj_qkvz | ||
| ) | ||
| self.in_proj_ba.weight.weight_loader = self._make_packed_weight_loader( | ||
| self.in_proj_ba.weight._weight_loader = self._make_packed_weight_loader( | ||
| self.in_proj_ba | ||
| ) |
There was a problem hiding this comment.
Directly assigning to a 'private' attribute like _weight_loader breaks encapsulation and makes the code brittle against dependency updates, as it relies on internal implementation details that could change without notice. A more robust approach is to use a public API if one is available for setting the weight loader. If no such public API exists, this workaround introduces a maintainability risk and it would be beneficial to add a comment explaining the need for this direct access for future reference.
|
/tag-and-rerun-ci |
|
/rerun-failed-ci |
|
ci 没过就给合了？ |
|
Seems breaking the qwen3-next accuracy |
This reverts commit 0906e45.
Motivation
previously,
When loading quantization weight such as w8a8, as weight_loader of self.in_proj_qkvz.weight is denoted by @Property, which means it doesn't have a setter method.
when running through this code and sequentially trying to assign another implementation for this method, the error occurs
Modifications
as follows.
Accuracy Tests
Benchmarking and Profiling
Checklist
Review Process
/tag-run-ci-label,/rerun-failed-ci,/tag-and-rerun-ci