Shortcuts

IoPathFileLister

class torchdata.datapipes.iter.IoPathFileLister(root: Union[str, Sequence[str], IterDataPipe], masks: Union[str, List[str]] = '', *, pathmgr=None)

Lists the contents of the directory at the provided root pathname or URL, and yields the full pathname or URL for each file within the directory (functional name: list_files_by_iopath).

Parameters:
  • root – The root local filepath or URL directory or list of roots to list files from

  • masks – Unix style filter string or string list for filtering file name(s)

  • pathmgr – Custom iopath.PathManager. If not specified, a default PathManager is created.

Note

Default PathManager currently supports local file path, normal HTTP URL and OneDrive URL. S3 URL is supported only with iopath>=0.1.9.

Example

>>> from torchdata.datapipes.iter import IoPathFileLister
>>> datapipe = IoPathFileLister(root=S3URL)

Docs

Access comprehensive developer documentation for PyTorch

View Docs

Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

Resources

Find development resources and get your questions answered

View Resources