FlatMapper

class torchdata.datapipes.iter.FlatMapper(datapipe: IterDataPipe, fn: Callable, input_col=None)

Applies a function over each item from the source DataPipe, then flattens the outputs to a single, unnested IterDataPipe (functional name: flatmap).

Note

The output from fn must be a Sequence. Otherwise, an error will be raised.

Parameters:

• datapipe – Source IterDataPipe

• fn – the function to be applied to each element in the DataPipe, the output must be a Sequence

Example

>>> from torchdata.datapipes.iter import IterableWrapper
>>> def fn(e):
>>>     return [e, e * 10]
>>> source_dp = IterableWrapper(list(range(5)))
>>> flatmapped_dp = source_dp.flatmap(fn)
>>> list(flatmapped_dp)
[0, 0, 1, 10, 2, 20, 3, 30, 4, 40]