MultiProcessingReadingService

class torchdata.dataloader2.MultiProcessingReadingService(num_workers: int = 0, pin_memory: bool = False, timeout: float = 0, worker_init_fn: Optional[Callable[[int], None]] = None, multiprocessing_context=None, prefetch_factor: int = 2, persistent_workers: bool = False)

MultiProcessingReadingService that utilizes torch.utils.data.DataLoader to launch subprocesses for DataPipe graph. Please refers to documents of DataLoader in https://pytorch.org/docs/stable/data.html#torch.utils.data.DataLoader for all arguments.

Note

This ReadingService be replaced by PrototypeMultiProcessingReadingService.

finalize() → None

ReadingService cleans up internal states and fully shuts down the service. Called in DataLoader2’s shutdown and __del__.

initialize(datapipe: Union[IterDataPipe, MapDataPipe]) → Union[IterDataPipe, MapDataPipe]

ReadingService takes a DataPipe graph, adapts it into a new DataPipe graph based on the custom need. Called once in creating DataLoader2 iterator at first time.

Parameters:

datapipe – Original DataPipe graph.

Returns:

An adapted or a new DataPipe graph.