Shortcuts

Forker

class torchdata.datapipes.iter.Forker(datapipe: IterDataPipe, num_instances: int, buffer_size: int = 1000)

Creates multiple instances of the same Iterable DataPipe (functional name: fork).

Parameters:
  • datapipe – Iterable DataPipe being copied

  • num_instances – number of instances of the datapipe to create

  • buffer_size – this restricts how far ahead the leading child DataPipe can read relative to the slowest child DataPipe. Defaults to 1000. Use -1 for the unlimited buffer.

Example

>>> # xdoctest: +REQUIRES(module:torchdata)
>>> from torchdata.datapipes.iter import IterableWrapper
>>> source_dp = IterableWrapper(range(5))
>>> dp1, dp2 = source_dp.fork(num_instances=2)
>>> list(dp1)
[0, 1, 2, 3, 4]
>>> list(dp2)
[0, 1, 2, 3, 4]

Docs

Access comprehensive developer documentation for PyTorch

View Docs

Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

Resources

Find development resources and get your questions answered

View Resources