[Fix] Add EPLB rebalance support for Kimi K2.5#21004
[Fix] Add EPLB rebalance support for Kimi K2.5#21004Kangyan-Zhou merged 1 commit intosgl-project:mainfrom
Conversation
Signed-off-by: yafeng.li <yafeng.li@mthreads.com>
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request addresses a critical Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
There was a problem hiding this comment.
Code Review
The pull request adds the routed_experts_weights_of_layer property to KimiK25ForConditionalGeneration to fix an AttributeError related to EPLB rebalance support. The change is straightforward and correctly delegates the property access to the underlying language model. I have one suggestion to improve code consistency by adding a return type hint to the new property.
| return self.language_model.end_layer | ||
|
|
||
| @property | ||
| def routed_experts_weights_of_layer(self): |
There was a problem hiding this comment.
For consistency with other properties in this class (like start_layer and end_layer) and for improved code clarity, please add a return type hint. Since this property appears to return a dictionary, -> dict would be an appropriate type hint.
| def routed_experts_weights_of_layer(self): | |
| def routed_experts_weights_of_layer(self) -> dict: |
|
Could you please take a look at this PR when you have a moment? Thanks! @yeahdongcn |
|
/tag-run-ci-label |
|
/tag-and-rerun-ci |
Signed-off-by: yafeng.li <yafeng.li@mthreads.com>
Signed-off-by: yafeng.li <yafeng.li@mthreads.com>
|
|
Could you share the error logs or stack trace when the server crashes? Thanks. |
I tested it locally and everything works fine. 
image: nvidia-smi: |
|
[2026-04-10 15:39:20 TP0 EP0] Decode batch, #running-req: 1, #token: 1938, token usage: 0.00, cuda graph: True, gen throughput (token/s): 94.64, #queue-req: 0 this is error logs with 0.5.10.post1, thanks @yafengio . |
Signed-off-by: yafeng.li <yafeng.li@mthreads.com>
Motivation
Add routed_experts_weights_of_layer property to KimiK25ForConditionalGeneration to enable EPLB (Expert Parallel Load Balancing) rebalance support for Kimi K2.5 models.
EPLB Rebalance error：
Modifications
Added
routed_experts_weights_of_layerproperty in KimiK25ForConditionalGeneration class.Accuracy Tests
Benchmarking and Profiling
Checklist
Review Process
/tag-run-ci-label,/rerun-failed-ci,/tag-and-rerun-ci