Flattener¶
- class torchdata.datapipes.iter.Flattener(datapipe: IterDataPipe, indices: Optional[Union[Hashable, List[Hashable]]] = None)¶
returns a flattened copy of the input DataPipe at the per sample/element level based on provided indices (functional name:
flatten
).Note
no args will flatten each item in the datapipe 1 level
- Parameters:
datapipe – IterDataPipe with iterable elements
indices – a single index/key for the item to flatten from an iterator item or a list of indices/keys to be flattened
Example
>>> from torchdata.datapipes.iter import IterableWrapper >>> dp = IterableWrapper([(0, 10, (100, 1000)), (1, 11, (111, 1001)), (2, 12, (122, 1002)), (3, 13, (133, 1003)), (4, 14, (144, 1004))]) >>> flatten_dp = dp.flatten(2) >>> list(flatten_dp) [(0, 10, 100, 1000), (1, 11, 111, 1001), (2, 12, 122, 1002), (3, 13, 133, 1003), (4, 14, 144, 1004)] >>> >>> dp = IterableWrapper([(0, (1, 2)), (3, (4, 5)), (6, (7, 8))]) >>> flatten_dp = dp.flatten() >>> list(flatten_dp) [(0, 1, 2), (3, 4, 5), (6, 7, 8)]