Skip to content

[DataLoader] Rename Functional DataSet to DataPipe#51488

Closed
ejguan wants to merge 3 commits intogh/ejguan/23/basefrom
gh/ejguan/23/head
Closed

[DataLoader] Rename Functional DataSet to DataPipe#51488
ejguan wants to merge 3 commits intogh/ejguan/23/basefrom
gh/ejguan/23/head

Conversation

@ejguan
Copy link
Copy Markdown
Contributor

@ejguan ejguan commented Feb 1, 2021 •
edited
Loading

Stack from ghstack:

Differential Revision: D26209888

This was referenced Feb 1, 2021
Closed
Closed
@facebook-github-bot
Copy link
Copy Markdown
Contributor

facebook-github-bot commented Feb 1, 2021 •
edited
Loading

💊 CI failures summary and remediations

As of commit ffff03f (more details on the Dr. CI page):

❄️ 1 failure tentatively classified as flaky

but reruns have not yet been triggered to confirm:

See CircleCI build pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_test2 (1/1)

Step: "Run tests" (full log | diagnosis details | 🔁 rerun) ❄️

Feb 03 02:32:55 RuntimeError: Process 0 terminated or timed out after 100.03785133361816 seconds 
Feb 03 02:32:55 ERROR: test_pytorch_parity (__main__.TestZeroRedundancyOptimizerDistributed)
Feb 03 02:32:55 When combined with DDP, check that ZeroRedundancyOptimizer(optimizer) and the same monolithic optimizer
Feb 03 02:32:55 ----------------------------------------------------------------------
Feb 03 02:32:55 Traceback (most recent call last):
Feb 03 02:32:55   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_distributed.py", line 282, in wrapper
Feb 03 02:32:55     self._join_processes(fn)
Feb 03 02:32:55   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_distributed.py", line 399, in _join_processes
Feb 03 02:32:55     self._check_return_codes(elapsed_time)
Feb 03 02:32:55   File "/opt/conda/lib/python3.6/site-packages/torch/testing/_internal/common_distributed.py", line 440, in _check_return_codes
Feb 03 02:32:55     raise RuntimeError('Process {} terminated or timed out after {} seconds'.format(i, elapsed_time))
Feb 03 02:32:55 RuntimeError: Process 0 terminated or timed out after 100.03785133361816 seconds
Feb 03 02:32:55 
Feb 03 02:32:55 ----------------------------------------------------------------------
Feb 03 02:32:55 Ran 12 tests in 131.572s
Feb 03 02:32:55 
Feb 03 02:32:55 FAILED (errors=1)
Feb 03 02:32:55 Traceback (most recent call last):
Feb 03 02:32:55   File "test/run_test.py", line 925, in <module>
Feb 03 02:32:55     main()
Feb 03 02:32:55   File "test/run_test.py", line 904, in main
Feb 03 02:32:55     raise RuntimeError(err_message)

ci.pytorch.org: 1 failed

This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

ejguan added a commit that referenced this pull request Feb 1, 2021
@ejguan
[DataLoader] Rename Functional DataSet to DataPipe
36da4b2 
ghstack-source-id: 8aced83
Pull Request resolved: #51488
@ejguan ejguan mentioned this pull request Feb 1, 2021
Closed
glaringlee
glaringlee approved these changes Feb 2, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@glaringlee glaringlee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, some nit comments, please update when rebasing the code.

Comment thread torch/utils/data/datapipes/iter/batch.py Outdated
Comment thread torch/utils/data/datapipes/iter/batch.py Outdated
Comment thread torch/utils/data/datapipes/iter/callable.py Outdated
Comment thread torch/utils/data/datapipes/iter/callable.py Outdated
Comment thread torch/utils/data/datapipes/iter/sampler.py
ejguan added a commit that referenced this pull request Feb 2, 2021
@ejguan
[DataLoader] Rename Functional DataSet to DataPipe
d739c11 
ghstack-source-id: 58df08d
Pull Request resolved: #51488
@facebook-github-bot
Copy link
Copy Markdown
Contributor

facebook-github-bot commented Feb 3, 2021

@ejguan merged this pull request in 52de407.

@facebook-github-bot facebook-github-bot deleted the gh/ejguan/23/head branch February 6, 2021 15:15
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 24, 2026
@ejguan @facebook-github-bot
[DataLoader] Rename Functional DataSet to DataPipe (pytorch#51488)
aadc281 
Summary: Pull Request resolved: pytorch#51488

Test Plan: Imported from OSS

Reviewed By: H-Huang

Differential Revision: D26209888

Pulled By: ejguan

fbshipit-source-id: cb8bc852b1e4d72be81e0297308a43954cd95332
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@glaringlee glaringlee glaringlee approved these changes

@VitalyFedyunin VitalyFedyunin Awaiting requested review from VitalyFedyunin

Assignees

No one assigned

Labels

cla signed Merged

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@ejguan @facebook-github-bot @glaringlee