FSSpecFileLister

class torchdata.datapipes.iter.FSSpecFileLister(root: str, masks: Union[str, List[str]] = '')

Lists the contents of the directory at the provided root pathname or URL, and yields the full pathname or URL for each file within the directory.

Parameters

• root – The root fsspec path directory to list files from

• masks – Unix style filter string or string list for filtering file name(s)

Example

>>> from torchdata.datapipes.iter import FSSpecFileLister
>>> datapipe = FSSpecFileLister(root=dir_path)