UnZipper¶
- class torchdata.datapipes.map.UnZipper(source_datapipe: MapDataPipe[Sequence[T]], sequence_length: int, columns_to_skip: Optional[Sequence[int]] = None)¶
Takes in a DataPipe of Sequences, unpacks each Sequence, and return the elements in separate DataPipes based on their position in the Sequence (functional name:
unzip
). The number of instances produced equals to thesequence_legnth
minus the number of columns to skip.Note
Each sequence within the DataPipe should have the same length, specified by the input argument sequence_length.
- Parameters:
source_datapipe – Iterable DataPipe with sequences of data
sequence_length – Length of the sequence within the source_datapipe. All elements should have the same length.
columns_to_skip – optional indices of columns that the DataPipe should skip (each index should be an integer from 0 to sequence_length - 1)
Example
>>> from torchdata.datapipes.iter import SequenceWrapper >>> source_dp = SequenceWrapper([(i, i + 10, i + 20) for i in range(3)]) >>> dp1, dp2, dp3 = source_dp.unzip(sequence_length=3) >>> list(dp1) [0, 1, 2] >>> list(dp2) [10, 11, 12] >>> list(dp3) [20, 21, 22]