Template Class StatefulDataset

• Defined in File stateful.h

Inheritance Relationships

Base Type

• public BatchDataset< Self, std::optional< std::vector< Example<>> >, size_t >

Class Documentation

template<typename Self, typename Batch = std::vector<Example<>>, typename BatchRequest = size_t>
class StatefulDataset : public BatchDataset<Self, std::optional<std::vector<Example<>>>, size_t>

A stateful dataset is a dataset that maintains some internal state, which will be reset() at the beginning of each epoch.

Subclasses can override the reset() method to configure this behavior. Further, the return type of a stateful dataset’s get_batch() method is always an optional. When the stateful dataset wants to indicate to the dataloader that its epoch has ended, it should return an empty optional. The dataloader knows to modify its implementation based on whether the dataset is stateless or stateful.

Note that when subclassing a from StatefulDataset<Self, T>, the return type of get_batch(), which the subclass must override, will be optional<T> (i.e. the type specified in the StatefulDataset specialization is automatically boxed into an optional for the dataset’s BatchType).

Public Functions

virtual void reset() = 0

Resets internal state of the dataset.

virtual void save(serialize::OutputArchive &archive) const = 0

Saves the statefulDataset’s state to OutputArchive.

virtual void load(serialize::InputArchive &archive) = 0

Deserializes the statefulDataset’s state from the archive.