Shortcuts

Class SequentialImpl

Inheritance Relationships

Base Type

Class Documentation

class torch::nn::SequentialImpl : public torch::nn::Cloneable<SequentialImpl>

A list of Modules that acts as a Module itself.

A Sequential is fundamentally a list of Modules, each with a forward() method. Sequential provides a forward() method of its own, which accepts any input and forwards it to the first module it stores. It then “chains” outputs to inputs sequentially for each subsequent module, finally returning the output of the last module. For example:

torch::nn::Sequential seq(
  torch::nn::Linear(3, 4),
  torch::nn::BatchNorm1d(4),
  torch::nn::Dropout(0.5)
);

auto output = seq->forward(torch::ones(3));

This can conceptually be thought of as the following loop (using Python as pseudocode):

def forward(sequential, input):
  for module in sequential:
    input = module(input)
  return input

Why should you use Sequential instead of a simple std::vector? The value a Sequential provides over manually calling a sequence of modules is that it allows treating the whole container as a single module, such that performing a transformation on the Sequential applies to each of the modules it stores (which are each a registered submodule of the Sequential). For example, calling .to(torch::kCUDA) on a Sequential will move each module in the list to CUDA memory. For example:

torch::nn::Sequential seq(
  torch::nn::Linear(3, 4),
  torch::nn::BatchNorm1d(4),
  torch::nn::Dropout(0.5)
);

// Convert all modules to CUDA.
seq->to(torch::kCUDA);

Finally, Sequential provides a lightweight container API, such as allowing iteration over submodules, positional access, adding a new module after construction via push_back, as well as joining two Sequentials via extend.

Attention

One current limitation of Sequential is that all except the first module must accept a single argument. If your modules need to take multiple arguments, you should define them to take and return tuples.

Public Types

using Iterator = std::vector<AnyModule>::iterator
using ConstIterator = std::vector<AnyModule>::const_iterator

Public Functions

SequentialImpl() = default
template<typename ...Modules>
SequentialImpl(Modules&&... modules)

Constructs the Sequential from a variadic list of modules.

SequentialImpl(torch::OrderedDict<std::string, AnyModule> &&ordered_dict)

Constructs the Sequential from an OrderedDict of named AnyModules.

SequentialImpl(std::initializer_list<NamedAnyModule> named_modules)

Constructs the Sequential from a braced-init-list of named AnyModules.

It enables the following use case: Sequential sequential({{"m1", M(1)}, {"m2", M(2)}})

std::shared_ptr<Module> clone(const optional<Device> &device = nullopt) const override

Special cloning function for Sequential because it does not use reset().

void reset() override

reset() is empty for Sequential, since it does not have parameters of its own.

void pretty_print(std::ostream &stream) const override

Pretty prints the Sequential module into the given stream.

template<typename ReturnType = Tensor, typename ...InputTypes>
ReturnType forward(InputTypes&&... inputs)

Feeds inputs to the first module and then chains outputs to inputs, returning the last output.

Conceptually the following loop in Python:

def forward(sequential, input):
  for module in sequential:
    input = module(input)
  return input

The return type is taken as the first template parameter. It defaults to Tensor. If the last module in the Sequential returns another type T, you should call forward<T>(inputs) instead of just forward(inputs):

torch::Tensor tensor = sequential1->forward(inputs);
int integer = sequential2->forward<int>(inputs);
float value = sequential3->forward<float>(inputs);

template<typename ModuleType>
void push_back(std::shared_ptr<ModuleType> module_ptr)

Adds a new (boxed) Module to the Sequential container.

template<typename ModuleType>
void push_back(std::string name, std::shared_ptr<ModuleType> module_ptr)

Adds a new named (boxed) Module to the Sequential container.

template<typename M, typename = torch::detail::enable_if_module_t<M>>
void push_back(M &&module)

Adds a new Module to the Sequential container, moving or copying it into a shared_ptr internally.

This method allows passing value types, and letting the container deal with the boxing. This means you can write Sequential(Module(3, 4)) instead of Sequential(std::make_shared<Module>(3, 4)).

template<typename M, typename = torch::detail::enable_if_module_t<M>>
void push_back(std::string name, M &&module)

Adds a new named Module to the Sequential container, moving or copying it into a shared_ptr internally.

This method allows passing value types, and letting the container deal with the boxing.

template<typename M>
void push_back(const ModuleHolder<M> &module_holder)

Unwraps the contained module of a ModuleHolder and adds it to the Sequential.

template<typename M>
void push_back(std::string name, const ModuleHolder<M> &module_holder)

Unwraps the contained named module of a ModuleHolder and adds it to the Sequential.

template<typename Container>
void extend(const Container &container)

Iterates over the container and calls push_back() on each value.

void push_back(AnyModule any_module)

Adds a type-erased AnyModule to the Sequential.

void push_back(std::string name, AnyModule any_module)
Iterator begin()

Returns an iterator to the start of the Sequential.

ConstIterator begin() const

Returns a const iterator to the start of the Sequential.

Iterator end()

Returns an iterator to the end of the Sequential.

ConstIterator end() const

Returns a const iterator to the end of the Sequential.

template<typename T>
T &at(size_t index)

Attempts to return the module at the given index as the requested type.

Throws an exception if the index is out of bounds or the types do not match.

template<typename T>
const T &at(size_t index) const

Attempts to return the module at the given index as the requested type.

Throws an exception if the index is out of bounds or the types do not match.

std::shared_ptr<Module> ptr(size_t index) const

Attempts to return a std::shared_ptr whose dynamic type is that of the underlying module at the given index.

Throws an exception if the index is out of bounds.

template<typename T>
std::shared_ptr<T> ptr(size_t index) const

Attempts to return a std::shared_ptr whose type is the one provided.

Throws an exception if the index is out of bounds or the types do not match.

std::shared_ptr<Module> operator[](size_t index) const

Like ptr(index).

size_t size() const noexcept

The current size of the Sequential container.

bool is_empty() const noexcept

True if there are no modules in the Sequential.

Docs

Access comprehensive developer documentation for PyTorch

View Docs

Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

Resources

Find development resources and get your questions answered

View Resources