MapKeyZipper¶
- class torchdata.datapipes.iter.MapKeyZipper(source_iterdatapipe: IterDataPipe, map_datapipe: MapDataPipe, key_fn: Callable, merge_fn: Optional[Callable] = None)¶
Joins the items from the source IterDataPipe with items from a MapDataPipe (functional name:
zip_with_map
). The matching is done by the providedkey_fn
, which maps an item fromsource_iterdatapipe
to a key that should exist in themap_datapipe
. The return value is created by themerge_fn
, which returns a tuple of the two items by default.- Parameters:
source_iterdatapipe – IterDataPipe from which items are yield and will be combined with an item from
map_datapipe
map_datapipe – MapDataPipe that takes a key from
key_fn
, and returns an itemkey_fn – Function that maps each item from
source_iterdatapipe
to a key that exists inmap_datapipe
merge_fn – Function that combines the item from
source_iterdatapipe
and the matching item frommap_datapipe
, by default a tuple is created
Example
>>> from torchdata.datapipes.iter import IterableWrapper >>> from torchdata.datapipes.map import SequenceWrapper >>> from operator import itemgetter >>> def merge_fn(tuple_from_iter, value_from_map): >>> return tuple_from_iter[0], tuple_from_iter[1] + value_from_map >>> dp1 = IterableWrapper([('a', 1), ('b', 2), ('c', 3)]) >>> mapdp = SequenceWrapper({'a': 100, 'b': 200, 'c': 300, 'd': 400}) >>> res_dp = dp1.zip_with_map(map_datapipe=mapdp, key_fn=itemgetter(0), merge_fn=merge_fn) >>> list(res_dp) [('a', 101), ('b', 202), ('c', 303)]