[2/n] Add setting function to set seeds to the graph#894
[2/n] Add setting function to set seeds to the graph#894ejguan wants to merge 4 commits intometa-pytorch:mainfrom
Conversation
|
@ejguan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
| return datapipe | ||
|
|
||
|
|
||
| def _set_graph_seeds(datapipe: DataPipe, seed_generator: torch.Generator, worker_id: int = 0) -> DataPipe: |
There was a problem hiding this comment.
I am using a private name because I don't like the name of this function. LMK if you have any better idea.
There was a problem hiding this comment.
_set_worker_seed_set_worker_seed_for_dp_graph
| return datapipe | ||
|
|
||
|
|
||
| def _set_graph_seeds(datapipe: DataPipe, seed_generator: torch.Generator, worker_id: int = 0) -> DataPipe: |
There was a problem hiding this comment.
_set_worker_seed_set_worker_seed_for_dp_graph
db08b88 to
fd7808a
Compare
|
@ejguan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@ejguan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
@ejguan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
NivekT
left a comment
There was a problem hiding this comment.
The BFS looks good to me. Can we add a test that confirms the result's ordering? (It might already exist.)
|
@ejguan has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
Changes
list_dpswith a new argument (to make sure allexclusive)DataPipeswill be excluded even though they are the predecessors of the non-excludedDataPipe.list_dpsto use BFS to return the list ofDataPipes._set_worker_seed_for_dp_graphto set seeds forDataPipesbeforesharding_filterwith the same seed generator and set different seeds forDataPipesaftersharding_filterwith the worker-local seed generator.Step 2 for #885