[feat] add memory saver support to sglang rollout#756
[feat] add memory saver support to sglang rollout#756eric-haibin-lin merged 11 commits intoverl-project:mainfrom
Conversation
eric-haibin-lin
left a comment
There was a problem hiding this comment.
could u check the test failure?
It seems due to lack of new dependency |
|
Please set memory saver to |
I will update the image today |
|
|
|
@vermouth1992 @PeterSH6 Hi team! I think we have the memory saver for SGLang ready to merge. Could you help trigger the CI? cc @zhaochenyang20 |
|
|
||
| logger = init_logger(__name__) | ||
| """ | ||
| This version is strongly tied with Megatron to implement HybridEngine and weight sharing between vllm and Megatron. |
There was a problem hiding this comment.
BTW i think the documentation requires updates too~
But it's okay to do it in upcoming PRs
|
thanks！ |
#756) as title --------- Co-authored-by: ocss884 <ocss.lin@gmail.com>
verl-project#756) as title --------- Co-authored-by: ocss884 <ocss.lin@gmail.com>
verl-project#756) as title --------- Co-authored-by: ocss884 <ocss.lin@gmail.com>
verl-project#756) as title --------- Co-authored-by: ocss884 <ocss.lin@gmail.com>
verl-project#756) as title --------- Co-authored-by: ocss884 <ocss.lin@gmail.com>
verl-project#756) as title --------- Co-authored-by: ocss884 <ocss.lin@gmail.com>
as title