torchtnt.utils.data.MultiDataLoader

class torchtnt.utils.data.MultiDataLoader(individual_dataloaders: Dict[str, Union[DataLoader, Iterable]], iteration_strategy: DataIterationStrategy, iterator_cls: Optional[Type[MultiIterator]] = None, ignore_empty_data: bool = False)

MultiDataLoader cycles through individual dataloaders passed to it.

individual_dataloaders

A dictionary of DataLoaders or Iterables with dataloader name as key

Type:Dict[str, Union[DataLoader, Iterable]]

and dataloader/iterable object as value.

iteration_strategy

A dataclass indicating how the dataloaders are iterated over.

Type:DataIterationStrategy

iterator_cls

A subclass of MultiIterator defining iteration logic. This is the type, not an object instance

Type:MultiIterator, optional

ignore_empty_data

skip dataloaders which contain no data. It’s False by default, and an exception is raised.

Type:bool

Note

TorchData also has generic multi-data sources reading support to achieve the same functionality provided by MultiIterator. For example, mux, mux_longest, cycle, zip etc. Please refer to the documentation for more details.

__init__(individual_dataloaders: Dict[str, Union[DataLoader, Iterable]], iteration_strategy: DataIterationStrategy, iterator_cls: Optional[Type[MultiIterator]] = None, ignore_empty_data: bool = False) → None

Methods

__init__(individual_dataloaders, ...[, ...])
load_state_dict(state_dict)	Loads aggregated state dict based on individual dataloaders.
state_dict()	Return an aggregated state dict based on individual dataloaders.