Skip to content

Prefix DataLoaderIter with underscore to discourage subclassing#5619

Merged
apaszke merged 1 commit intopytorch:masterfrom
ssnl:dlset
Mar 8, 2018
Merged

Prefix DataLoaderIter with underscore to discourage subclassing#5619
apaszke merged 1 commit intopytorch:masterfrom
ssnl:dlset

Conversation

@ssnl
Copy link
Copy Markdown
Collaborator

@ssnl ssnl commented Mar 7, 2018

Discourage subclassing DataLoaderIter so that errors like #5430 hopefully won't happen as often.

@apaszke

@apaszke apaszke merged commit 04461fa into pytorch:master Mar 8, 2018
@ssnl ssnl deleted the dlset branch March 8, 2018 21:18
mdraw added a commit to ELEKTRONN/elektronn3 that referenced this pull request Mar 21, 2018
DataLoaderIter has been renamed to _DataLoaderIter "to discourage
subclassing" (pytorch/pytorch#5619).

The custom DelayededDataLoader mechanism is still useful for two reasons:
- Prevents crashes when KeyboardInterrupts are triggered
  "at the wrong time". Replacing everything with vanilla PyTorch
  DataLoaders leads to a complete crash when <C-c>-ing during background
  process initialization
  (RuntimeError: DataLoader worker (pid 15559) exited unexpectedly with
   exit code 1).
  But:
  When initialization is finished (so, after ~2 seconds), the vanilla
  PyTorch DataLoader does recover from interrupts: When continuing, new
  background processes are started automatically and continue making batches.
- Reduced visual noise in the terminal (large tracebacks are suppressed)

I am not sure if those reasons are good enough for keeping it though...
laurentdupin pushed a commit to laurentdupin/pytorch that referenced this pull request Apr 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants