class torchdata.dataloader2.InProcessReadingService(prefetch_cnt: int = 0, init_fn: Optional[Callable[[Union[IterDataPipe, MapDataPipe], WorkerInfo], Union[IterDataPipe, MapDataPipe]]] = None, reset_fn: Optional[Callable[[Union[IterDataPipe, MapDataPipe], WorkerInfo, SeedGenerator], Union[IterDataPipe, MapDataPipe]]] = None)

Default ReadingService to serve the ``DataPipe` graph in the main process, and apply graph settings like determinism control to the graph.

  • prefetch_cnt – (int, 0 by default): Number of data will be prefetched in the main process.

  • init_fn – (Callable, optional): Custom function to be called when the main process starts to iterate over DataPipe graph.

  • reset_fn – (Callable, optional): Custom function to be called at the beginning of each epoch with DataPipe, WorkerInfo and SeedGenerator as the expected arguments.

initialize(datapipe: Union[IterDataPipe, MapDataPipe]) Union[IterDataPipe, MapDataPipe]

ReadingService takes a DataPipe graph, adapts it into a new DataPipe graph based on the custom need. Called once in creating DataLoader2 iterator at first time. Prior to calling this method, the ReadingService object must be picklable.


datapipe – Original DataPipe graph.


An adapted or a new DataPipe graph.

initialize_iteration(seed_generator: SeedGenerator, iter_reset_fn: Optional[Callable[[Union[IterDataPipe, MapDataPipe]], Union[IterDataPipe, MapDataPipe]]] = None) Optional[Callable[[Union[IterDataPipe, MapDataPipe]], Union[IterDataPipe, MapDataPipe]]]

ReadingService spins up service for an epoch. Called at the beginning of every time getting DataLoader2 iterator.

  • seed_generator – SeedGenerator object created and managed by DataLoader2. As the single source of randomness, it will govern the determinism for all of random operations with the graph of DataPipes.

  • iter_reset_fn – Optional reset function from the prior ReadingServcie when SequentialReadingService chains multiple ReadingServices


A new iter_reset_fn to be used by subseqeuent ReadingService


MultiProcessingReadingService starts setting worker seeds per process and prefetching items from the graph.


Access comprehensive developer documentation for PyTorch

View Docs


Get in-depth tutorials for beginners and advanced developers

View Tutorials


Find development resources and get your questions answered

View Resources