Forker

class torchdata.datapipes.iter.Forker(datapipe: IterDataPipe, num_instances: int, buffer_size: int = 1000)

Creates multiple instances of the same Iterable DataPipe (functional name: fork).

Parameters:

• datapipe – Iterable DataPipe being copied

• num_instances – number of instances of the datapipe to create

• buffer_size – this restricts how far ahead the leading child DataPipe can read relative to the slowest child DataPipe. Defaults to 1000. Use -1 for the unlimited buffer.

Example

>>> from torchdata.datapipes.iter import IterableWrapper
>>> source_dp = IterableWrapper(range(5))
>>> dp1, dp2 = source_dp.fork(num_instances=2)
>>> list(dp1)
[0, 1, 2, 3, 4]
>>> list(dp2)
[0, 1, 2, 3, 4]