StreamReader

class torchdata.datapipes.iter.StreamReader(datapipe, chunk=None)

Given IO streams and their label names, yields bytes with label name in a tuple (functional name: read_from_stream).

Parameters:

• datapipe – Iterable DataPipe provides label/URL and byte stream

• chunk – Number of bytes to be read from stream per iteration. If None, all bytes will be read until the EOF.

Example

>>> # xdoctest: +SKIP
>>> from torchdata.datapipes.iter import IterableWrapper, StreamReader
>>> from io import StringIO
>>> dp = IterableWrapper([("alphabet", StringIO("abcde"))])
>>> list(StreamReader(dp, chunk=1))
[('alphabet', 'a'), ('alphabet', 'b'), ('alphabet', 'c'), ('alphabet', 'd'), ('alphabet', 'e')]