IterToMapConverter¶
- class torchdata.datapipes.map.IterToMapConverter(datapipe: IterDataPipe, key_value_fn: Optional[Callable] = None)¶
Lazily load data from
IterDataPipe
to construct aMapDataPipe
with the key-value pair generated bykey_value_fn
(functional name:to_map_datapipe
). Ifkey_value_fn
is not given, each data from the source IterDataPipe must itself be an iterable with exactly two objects. The first object of each item becomes a key in the new dictionary, and the second object the corresponding value.For the opposite converter, use
MapToIterConverter
.- Parameters:
datapipe – Source IterDataPipe
key_value_fn – Function being applied over each data to generate key-value pair
Note
If a key being added is already present, the corresponding value will be replaced by the new value.
Example
>>> from torchdata.datapipes.iter import IterableWrapper >>> source_dp = IterableWrapper([(i, i) for i in range(10)]) >>> map_dp = source_dp.to_map_datapipe() >>> list(map_dp) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] >>> source_dp2 = IterableWrapper([('a', 1), ('b', 2), ('c', 1)]) >>> map_dp2 = source_dp2.to_map_datapipe() >>> map_dp2['a'] 1 >>> def row_to_tuple(row): >>> label = row[0] >>> data = row[1:] >>> return label, data >>> source_dp3 = IterableWrapper([('a', 1, 1, 1, 1, 1, 1), ('b', 2, 2, 2, 2, 2, 2), ('c', 3, 3, 3, 3, 3, 3)]) >>> map_dp3 = source_dp3.to_map_datapipe(key_value_fn=row_to_tuple) >>> map_dp3['a'] (1, 1, 1, 1, 1, 1)