Map-style DataPipes

A Map-style DataPipe is one that implements the __getitem__() and __len__() protocols, and represents a map from (possibly non-integral) indices/keys to data samples.

For example, when accessed with mapdatapipe[idx], could read the idx-th image and its corresponding label from a folder on the disk.

class*args, **kwds)

Map-style DataPipe.

All datasets that represent a map from keys to data samples should subclass this. Subclasses should overwrite __getitem__(), supporting fetching a data sample for a given, unique key. Subclasses can also optionally overwrite __len__(), which is expected to return the size of the dataset by many Sampler implementations and the default options of DataLoader.

These DataPipes can be invoked in two ways, using the class constructor or applying their functional form onto an existing MapDataPipe (recommend, available to most but not all DataPipes).


DataLoader by default constructs an index sampler that yields integral indices. To make it work with a map-style DataPipe with non-integral indices/keys, a custom sampler must be provided.


>>> from import SequenceWrapper, Mapper
>>> dp = SequenceWrapper(range(10))
>>> map_dp_1 = x: x + 1)  # Using functional form (recommended)
>>> list(map_dp_1)
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>> map_dp_2 = Mapper(dp, lambda x: x + 1)  # Using class constructor
>>> list(map_dp_2)
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>> batch_dp = map_dp_1.batch(batch_size=2)
>>> list(batch_dp)
[[1, 2], [3, 4], [5, 6], [7, 8], [9, 10]]

Here is the list of available Map-style DataPipes:



Create mini-batches of data (functional name: batch).


Concatenate multiple Map DataPipes (functional name: concat).


Lazily load data from IterDataPipe to construct a MapDataPipe with the key-value pair generated by key_value_fn (functional name: to_map_datapipe).


Apply the input function over each item from the source DataPipe (functional name: map).


Wraps a sequence object into a MapDataPipe.


Shuffle the input DataPipe via its indices (functional name: shuffle).


Aggregates elements into a tuple from each of the input DataPipes (functional name: zip).


Access comprehensive developer documentation for PyTorch

View Docs


Get in-depth tutorials for beginners and advanced developers

View Tutorials


Find development resources and get your questions answered

View Resources