UnZipper

class torchdata.datapipes.iter.UnZipper(source_datapipe: IterDataPipe[Sequence[T]], sequence_length: int, buffer_size: int = 1000, columns_to_skip: Optional[Sequence[int]] = None)

Takes in a DataPipe of Sequences, unpacks each Sequence, and return the elements in separate DataPipes based on their position in the Sequence (functional name: unzip). The number of instances produced equals to the sequence length minus the number of columns to skip.

Note

Each sequence within the DataPipe should have the same length, specified by the input argument sequence_length.

Parameters:

• source_datapipe – Iterable DataPipe with sequences of data

• sequence_length – Length of the sequence within the source_datapipe. All elements should have the same length.

• buffer_size – this restricts how far ahead the leading child DataPipe can read relative to the slowest child DataPipe. Use -1 for the unlimited buffer.

• columns_to_skip – optional indices of columns that the DataPipe should skip (each index should be an integer from 0 to sequence_length - 1)

Example

>>> from torchdata.datapipes.iter import IterableWrapper
>>> source_dp = IterableWrapper([(i, i + 10, i + 20) for i in range(3)])
>>> dp1, dp2, dp3 = source_dp.unzip(sequence_length=3)
>>> list(dp1)
[0, 1, 2]
>>> list(dp2)
[10, 11, 12]
>>> list(dp3)
[20, 21, 22]