You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There is a number token and request level memory accounting issues reported and fixed by the community recently. It would be good to complement them with robust CI tests.
Checklist
Motivation
There is a number token and request level memory accounting issues reported and fixed by the community recently. It would be good to complement them with robust CI tests.
Context:
sglang/python/sglang/srt/managers/schedule_policy.py
Lines 666 to 668 in 5ef7287
Related resources
No response