CacheTimeout

class torchdata.dataloader2.adapter.CacheTimeout(timeout=None)

CacheTimeout DataPipes adapter allows control over timeouts of all existing EndOnDiskCacheHolder (end_caching) in the graph. Useful when cached pipeline takes too long to execute (ex. slow file downloading).

Parameters:

timeout – int - amount of seconds parallel processes will wait for cached files to appear.

Example:

dp = IterableWrapper(range(size)).shuffle()
dl = DataLoader2(dp, [CacheTimeout(600)])