RarArchiveLoader¶
- class torchdata.datapipes.iter.RarArchiveLoader(datapipe: IterDataPipe[Tuple[str, BufferedIOBase]], *, length: int = -1)¶
Decompresses rar binary streams from input Iterable Datapipes which contains tuples of path name and rar binary stream, and yields a tuple of path name and extracted binary stream (functional name:
load_from_rar
).Note
The nested RAR archive is not supported by this DataPipe due to the limitation of the archive type. Please extract outer RAR archive before reading the inner archive.
- Parameters:
datapipe – Iterable DataPipe that provides tuples of path name and rar binary stream
length – Nominal length of the DataPipe
Example
>>> from torchdata.datapipes.iter import FileLister, FileOpener >>> datapipe1 = FileLister(".", "*.rar") >>> datapipe2 = FileOpener(datapipe1, mode="b") >>> rar_loader_dp = datapipe2.load_from_rar() >>> for _, stream in rar_loader_dp: >>> print(stream.read()) b'0123456789abcdef'