IterToMapConverter

class torchdata.datapipes.map.IterToMapConverter(datapipe: IterDataPipe, key_value_fn: Optional[Callable] = None)

Lazily load data from IterDataPipe to construct a MapDataPipe with the key-value pair generated by key_value_fn (functional name: to_map_datapipe). If key_value_fn is not given, each data from the source IterDataPipe must itself be an iterable with exactly two objects. The first object of each item becomes a key in the new dictionary, and the second object the corresponding value.

For the opposite converter, use MapToIterConverter.

Parameters:

• datapipe – Source IterDataPipe

• key_value_fn – Function being applied over each data to generate key-value pair

Note

If a key being added is already present, the corresponding value will be replaced by the new value.

Example

>>> from torchdata.datapipes.iter import IterableWrapper
>>> source_dp = IterableWrapper([(i, i) for i in range(10)])
>>> map_dp = source_dp.to_map_datapipe()
>>> list(map_dp)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> source_dp2 = IterableWrapper([('a', 1), ('b', 2), ('c', 1)])
>>> map_dp2 = source_dp2.to_map_datapipe()
>>> map_dp2['a']
1
>>> def row_to_tuple(row):
>>>     label = row[0]
>>>     data = row[1:]
>>>     return label, data
>>> source_dp3 = IterableWrapper([('a', 1, 1, 1, 1, 1, 1), ('b', 2, 2, 2, 2, 2, 2), ('c', 3, 3, 3, 3, 3, 3)])
>>> map_dp3 = source_dp3.to_map_datapipe(key_value_fn=row_to_tuple)
>>> map_dp3['a']
(1, 1, 1, 1, 1, 1)