Forker¶
- class torchdata.datapipes.iter.Forker(datapipe: IterDataPipe, num_instances: int, buffer_size: int = 1000)¶
Creates multiple instances of the same Iterable DataPipe (functional name:
fork
).- Parameters:
datapipe – Iterable DataPipe being copied
num_instances – number of instances of the datapipe to create
buffer_size – this restricts how far ahead the leading child DataPipe can read relative to the slowest child DataPipe. Defaults to
1000
. Use-1
for the unlimited buffer.
Example
>>> # xdoctest: +REQUIRES(module:torchdata) >>> from torchdata.datapipes.iter import IterableWrapper >>> source_dp = IterableWrapper(range(5)) >>> dp1, dp2 = source_dp.fork(num_instances=2) >>> list(dp1) [0, 1, 2, 3, 4] >>> list(dp2) [0, 1, 2, 3, 4]