curious: why demux is PyTorch core (https://github.com/pytorch/pytorch/blob/7a2930b357a4e62bb0bab53bb0d23c607b6ede38/torch/utils/data/datapipes/iter/combining.py#L305-L306) but round_robin_demux is in TorchData?

Historical reason. We put lots of DataPipe in core but we plan to move them to torchdata eventually. To reduce the amount of work, I think it's better to put round_robin_demux to torchdata directly.

As a Python noob, TIL I learnt X(...) in Python may not return a instance of X

https://www.geeksforgeeks.org/__new__-in-python/

... because other class constructors can be called by new method or it can simply return other objects as an instance of this class.

Will _ChildDataPipe.__init__ be called for each instance in the list? ~

Yeah.

curious: can we just do

_DemultiplexerIterDataPipe(
    datapipe, 
    num_instances, 
    partial(num_instances, _round_robin_fn), 
    False, 
    buffer_size
)

where _round_robin_fn is:

def _round_robin_fn(num_instances: int, idx_data) -> int:
        idx, _ = idx_data
        return idx % num_instances

Is that because you want to override _DemultiplexerIterDataPipe.get_length_by_instance ?

Yeah. demux doesn't provide a valid length but this DataPipe should have a deterministic result. Let me know if you have better way to do so.

noob question : what's "deterministic result" here mean?

Especially I was somehow under the impression that in general try to avoid query len(datapipe) for large datapipe , since it probably needs to read all data and it's expensive?

If the prior DataPipe provides a length, we should be able to get the length ahead of time.
For demux which uses a classify_fn, there is no way for us to know the length and the length can vary across epoches.

in general try to avoid query len(datapipe) for large datapipe , since it probably needs to read all data and it's expensive?

It's true. But, we do provide DataPipe that can manually assign length to the DataPipe https://github.com/pytorch/data/blob/67afe0c36f5f93d34fde7026e7e95835c1160bb3/torchdata/datapipes/iter/util/header.py#L62