Mapper¶
- class torchdata.datapipes.iter.Mapper(datapipe: IterDataPipe, fn: Callable, input_col=None, output_col=None)¶
Applies a function over each item from the source DataPipe (functional name:
map
). The function can be any regular Python function or partial object. Lambda function is not recommended as it is not supported by pickle.- Parameters:
datapipe – Source Iterable DataPipe
fn – Function being applied over each item
input_col –
Index or indices of data which
fn
is applied, such as:None
as default to applyfn
to the data directly.Integer(s) is used for list/tuple.
Key(s) is used for dict.
output_col –
Index of data where result of
fn
is placed.output_col
can be specified only wheninput_col
is notNone
None
as default to replace the index thatinput_col
specified; Forinput_col
with multiple indices, the left-most one is used, and other indices will be removed.Integer is used for list/tuple.
-1
represents to append result at the end.Key is used for dict. New key is acceptable.
Example
>>> # xdoctest: +SKIP >>> from torchdata.datapipes.iter import IterableWrapper, Mapper >>> def add_one(x): ... return x + 1 >>> dp = IterableWrapper(range(10)) >>> map_dp_1 = dp.map(add_one) # Invocation via functional form is preferred >>> list(map_dp_1) [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] >>> # We discourage the usage of `lambda` functions as they are not serializable with `pickle` >>> # Use `functools.partial` or explicitly define the function instead >>> map_dp_2 = Mapper(dp, lambda x: x + 1) >>> list(map_dp_2) [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]