RoundRobinDemultiplexer

class torchdata.datapipes.iter.RoundRobinDemultiplexer(datapipe: IterDataPipe, num_instances: int, buffer_size: int = 1000)

Splits the input DataPipe into multiple child DataPipes in the round-robin order (functional name: round_robin_demux). A list of the child DataPipes is returned from this operation.

Parameters:

• datapipe – Iterable DataPipe being filtered

• num_instances – number of instances of the DataPipe to create

• buffer_size – this defines the maximum number of inputs that the buffer can hold across all child DataPipes while waiting for their values to be yielded. Defaults to 1000. Use -1 for the unlimited buffer.

Examples

>>> from torchdata.datapipes.iter import IterableWrapper
>>> source_dp = IterableWrapper(range(5))
>>> dp1, dp2 = source_dp.round_robin_demux(2)
>>> list(dp1)
[0, 2, 4]
>>> len(dp1)
3
>>> list(dp2)
[1, 3]
>>> len(dp2)
2