Shortcuts

StreamReader

class torchdata.datapipes.iter.StreamReader(datapipe, chunk=None)

Given IO streams and their label names, yields bytes with label name in a tuple (functional name: read_from_stream).

Parameters:
  • datapipe – Iterable DataPipe provides label/URL and byte stream

  • chunk – Number of bytes to be read from stream per iteration. If None, all bytes will be read until the EOF.

Example

>>> # xdoctest: +SKIP
>>> from torchdata.datapipes.iter import IterableWrapper, StreamReader
>>> from io import StringIO
>>> dp = IterableWrapper([("alphabet", StringIO("abcde"))])
>>> list(StreamReader(dp, chunk=1))
[('alphabet', 'a'), ('alphabet', 'b'), ('alphabet', 'c'), ('alphabet', 'd'), ('alphabet', 'e')]

Docs

Access comprehensive developer documentation for PyTorch

View Docs

Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

Resources

Find development resources and get your questions answered

View Resources