GDriveReader¶
- class torchdata.datapipes.iter.GDriveReader(source_datapipe: IterDataPipe[str], *, timeout: Optional[float] = None)¶
Takes URLs pointing at GDrive files, and yields tuples of file name and IO stream (functional name:
read_from_gdrive
).- Parameters:
source_datapipe – a DataPipe that contains URLs to GDrive files
timeout – timeout in seconds for HTTP request
Example
>>> from torchdata.datapipes.iter import IterableWrapper, GDriveReader >>> gdrive_file_url = "https://drive.google.com/uc?export=download&id=SomeIDToAGDriveFile" >>> gdrive_reader_dp = GDriveReader(IterableWrapper([gdrive_file_url])) >>> reader_dp = gdrive_reader_dp.readlines() >>> it = iter(reader_dp) >>> path, line = next(it) >>> path https://drive.google.com/uc?export=download&id=SomeIDToAGDriveFile >>> line <First line from the GDrive File>