verl-project/verl#756

we will continue on this

FYI from the PyTorch side, it is clear that whether this is a bug or a feature is not a clear cut :D
We have users that depend on the current behavior and so this patch might have surprising behavior for them.

In the context of this project, I'm curious if finer grained control of CUDA_VISIBLE_DEVICES would allow you to avoid the issue?

Also it is still very unclear to me how you get in a situation where two gpus are on the same machine (as they get confused via CUDA_VISIBLE_DEVICES) but don't have peer access?

@albanD

We have users that depend on the current behavior and so this patch might have surprising behavior for them.

(replied in pytorch/pytorch#149248)

In the context of this project, I'm curious if finer grained control of CUDA_VISIBLE_DEVICES would allow you to avoid the issue?

Seems not very trivial for the current code (but the scenario may change in the future)

Also it is still very unclear to me how you get in a situation where two gpus are on the same machine (as they get confused via CUDA_VISIBLE_DEVICES) but don't have peer access?

Firstly, IIRC I do see some users report this, maybe some 4090s etc.

Secondly, it is because I do https://github.com/fzyzcjy/torch_memory_saver, and thus it is more sensitive to correct gpu devices. (I can also choose to enable peer access of all gpus but that would be slightly slower)

Seems not very trivial for the current code (but the scenario may change in the future)

That is unfortunate :/

Secondly, it is because I do https://github.com/fzyzcjy/torch_memory_saver, and thus it is more sensitive to correct gpu devices.

Very cool project :)
All the data is lost after a pause/resume though? CPU offloading the data could be a cool option !
Also we have https://docs.pytorch.org/docs/stable/notes/cuda.html#mixing-different-cuda-system-allocators-in-the-same-program that might be of interest for you for this kind of advanced usage.

Very cool project :)

Thank you :)

All the data is lost after a pause/resume though? CPU offloading the data could be a cool option !

Yes it can backup/restore data as well, will be on master branch later. It originally get lost b/c in RL people do not need it.

Also we have docs.pytorch.org/docs/stable/notes/cuda.html#mixing-different-cuda-system-allocators-in-the-same-program that might be of interest for you for this kind of advanced usage.

Yes I know it :) Choose the current one s.t. I can even release all pytorch low-level memory etc, and also use pytorch's caching allocator powerfulness.