Shortcuts

FSSpecFileLister

class torchdata.datapipes.iter.FSSpecFileLister(root: Union[str, Sequence[str], IterDataPipe], masks: Union[str, List[str]] = '', **kwargs)

Lists the contents of the directory at the provided root pathname or URL, and yields the full pathname or URL for each file within the directory (functional name: list_files_by_fsspec).

Parameters:
  • root – The root fsspec path directory or list of path directories to list files from

  • masks – Unix style filter string or string list for filtering file name(s)

  • kwargs – Extra options that make sense to a particular storage connection, e.g. host, port, username, password, etc.

Example

>>> from torchdata.datapipes.iter import FSSpecFileLister
>>> datapipe = FSSpecFileLister(root=dir_path)

Docs

Access comprehensive developer documentation for PyTorch

View Docs

Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

Resources

Find development resources and get your questions answered

View Resources