Class SequentialImpl

• Defined in File sequential.h

Inheritance Relationships

Base Type

• public torch::nn::Cloneable< SequentialImpl > (Template Class Cloneable)

Class Documentation

class SequentialImpl : public torch::nn::Cloneable<SequentialImpl>

A list of Modules that acts as a Module itself.

A Sequential is fundamentally a list of Modules, each with a forward() method. Sequential provides a forward() method of its own, which accepts any input and forwards it to the first module it stores. It then “chains” outputs to inputs sequentially for each subsequent module, finally returning the output of the last module. For example:

torch::nn::Sequential seq(
  torch::nn::Linear(3, 4),
  torch::nn::BatchNorm1d(4),
  torch::nn::Dropout(0.5)
);

auto output = seq->forward(torch::ones(3));

This can conceptually be thought of as the following loop (using Python as pseudocode):

def forward(sequential, input):
  for module in sequential:
    input = module(input)
  return input

Why should you use Sequential instead of a simple std::vector? The value a Sequential provides over manually calling a sequence of modules is that it allows treating the whole container as a single module, such that performing a transformation on the Sequential applies to each of the modules it stores (which are each a registered submodule of the Sequential). For example, calling .to(torch::kCUDA) on a Sequential will move each module in the list to CUDA memory. For example:

torch::nn::Sequential seq(
  torch::nn::Linear(3, 4),
  torch::nn::BatchNorm1d(4),
  torch::nn::Dropout(0.5)
);

// Convert all modules to CUDA.
seq->to(torch::kCUDA);

Finally, Sequential provides a lightweight container API, such as allowing iteration over submodules, positional access, adding a new module after construction via push_back, as well as joining two Sequentials via extend.

Attention

One current limitation of Sequential is that all except the first module must accept a single argument. If your modules need to take multiple arguments, you should define them to take and return tuples.

Public Types

using Iterator = std::vector<AnyModule>::iterator

using ConstIterator = std::vector<AnyModule>::const_iterator

Public Functions

SequentialImpl() = default

template<typename ...Modules>
inline explicit SequentialImpl(Modules&&... modules)

Constructs the Sequential from a variadic list of modules.

inline explicit SequentialImpl(torch::OrderedDict<std::string, AnyModule> &&ordered_dict)

Constructs the Sequential from an OrderedDict of named AnyModules.

inline explicit SequentialImpl(std::initializer_list<NamedAnyModule> named_modules)

Constructs the Sequential from a braced-init-list of named AnyModules.

It enables the following use case: Sequential sequential({{"m1", M(1)}, {"m2", M(2)}})

inline virtual std::shared_ptr<Module> clone(const std::optional<Device> &device = std::nullopt) const override

Special cloning function for Sequential because it does not use reset().

inline virtual void reset() override

reset() is empty for Sequential, since it does not have parameters of its own.

inline virtual void pretty_print(std::ostream &stream) const override

Pretty prints the Sequential module into the given stream.

template<typename ReturnType = Tensor, typename ...InputTypes>
inline ReturnType forward(InputTypes&&... inputs)

Feeds inputs to the first module and then chains outputs to inputs, returning the last output.

Conceptually the following loop in Python:

def forward(sequential, input):
  for module in sequential:
    input = module(input)
  return input

The return type is taken as the first template parameter. It defaults to Tensor. If the last module in the Sequential returns another type T, you should call forward<T>(inputs) instead of just forward(inputs):

torch::Tensor tensor = sequential1->forward(inputs);
int integer = sequential2->forward<int>(inputs);
float value = sequential3->forward<float>(inputs);

template<typename ModuleType>
inline void push_back(std::shared_ptr<ModuleType> module_ptr)

Adds a new (boxed) Module to the Sequential container.

template<typename ModuleType>
inline void push_back(std::string name, std::shared_ptr<ModuleType> module_ptr)

Adds a new named (boxed) Module to the Sequential container.

template<typename M, typename = torch::detail::enable_if_module_t<M>>
inline void push_back(M &&module)

Adds a new Module to the Sequential container, moving or copying it into a shared_ptr internally.

This method allows passing value types, and letting the container deal with the boxing. This means you can write Sequential(Module(3, 4)) instead of Sequential(std::make_shared<Module>(3, 4)).

template<typename M, typename = torch::detail::enable_if_module_t<M>>
inline void push_back(std::string name, M &&module)

Adds a new named Module to the Sequential container, moving or copying it into a shared_ptr internally.

This method allows passing value types, and letting the container deal with the boxing.

template<typename M>
inline void push_back(const ModuleHolder<M> &module_holder)

Unwraps the contained module of a ModuleHolder and adds it to the Sequential.

template<typename M>
inline void push_back(std::string name, const ModuleHolder<M> &module_holder)

Unwraps the contained named module of a ModuleHolder and adds it to the Sequential.

template<typename Container>
inline void extend(const Container &container)

Iterates over the container and calls push_back() on each value.

inline void push_back(AnyModule any_module)

Adds a type-erased AnyModule to the Sequential.

inline void push_back(std::string name, AnyModule any_module)

inline Iterator begin()

Returns an iterator to the start of the Sequential.

inline ConstIterator begin() const

Returns a const iterator to the start of the Sequential.

inline Iterator end()

Returns an iterator to the end of the Sequential.

inline ConstIterator end() const

Returns a const iterator to the end of the Sequential.

template<typename T>
inline T &at(size_t index)

Attempts to return the module at the given index as the requested type.

Throws an exception if the index is out of bounds or the types do not match.

template<typename T>
inline const T &at(size_t index) const

Attempts to return the module at the given index as the requested type.

Throws an exception if the index is out of bounds or the types do not match.

inline std::shared_ptr<Module> ptr(size_t index) const

Attempts to return a std::shared_ptr whose dynamic type is that of the underlying module at the given index.

Throws an exception if the index is out of bounds.

template<typename T>
inline std::shared_ptr<T> ptr(size_t index) const

Attempts to return a std::shared_ptr whose type is the one provided.

Throws an exception if the index is out of bounds or the types do not match.

inline std::shared_ptr<Module> operator[](size_t index) const

Like ptr(index).

inline size_t size() const noexcept

The current size of the Sequential container.

inline bool is_empty() const noexcept

True if there are no modules in the Sequential.