UnZipper

class torchdata.datapipes.map.UnZipper(source_datapipe: MapDataPipe[Sequence[T]], sequence_length: int, columns_to_skip: Optional[Sequence[int]] = None)

Takes in a DataPipe of Sequences, unpacks each Sequence, and return the elements in separate DataPipes based on their position in the Sequence (functional name: unzip). The number of instances produced equals to the sequence_legnth minus the number of columns to skip.

Note

Each sequence within the DataPipe should have the same length, specified by the input argument sequence_length.

Parameters:

• source_datapipe – Iterable DataPipe with sequences of data

• sequence_length – Length of the sequence within the source_datapipe. All elements should have the same length.

• columns_to_skip – optional indices of columns that the DataPipe should skip (each index should be an integer from 0 to sequence_length - 1)

Example

>>> from torchdata.datapipes.map import SequenceWrapper
>>> source_dp = SequenceWrapper([(i, i + 10, i + 20) for i in range(3)])
>>> dp1, dp2, dp3 = source_dp.unzip(sequence_length=3)
>>> list(dp1)
[0, 1, 2]
>>> list(dp2)
[10, 11, 12]
>>> list(dp3)
[20, 21, 22]