Struct InterpreterManager

• Defined in File deploy.h

Struct Documentation

struct InterpreterManager

An InterpreterManager handles the interaction of multiple subinterpreters such as allocating subinterpreters, or load balancing the subinterpreters.

Public Functions

explicit InterpreterManager(size_t nInterp = 2, std::shared_ptr<Environment> env = std::make_shared<NoopEnvironment>())

constructor for InterpreterManager which takes the number of interpreters (usually correlates to number of cores on your cpu), and a pointer to an Environment.

The default uses the local python env.

inline InterpreterSession acquireOne()

Returns a free interpreter or an arbitrary interpreter if there are none free.

To ensure data safety it’s best to match the number of calling threads to the size of the interpreter pool to avoid sharing an interpreter.

inline at::ArrayRef<Interpreter> allInstances()

use to make sure something gets run on all interpreters, such as loading or unloading a model eagerly

inline void debugLimitInterpreters(size_t N)

debugging tool to control the size of the loadBalancer and change the number of interpreters on the fly

Package loadPackage(const std::string &uri)

loads a package from a file with name uri

Package loadPackage(std::shared_ptr<caffe2::serialize::ReadAdapterInterface> reader)

loads a package from a PyTorchStreamReader or any class other which uses ReadAdapterInterface

inline void registerModuleSource(std::string name, std::string src)

convience function for loading some python source code as a module across all interpreters.

this can be used for writing tests of deploy that need to execute python code, or for small amounts of application logic that are best written in Python. For larger amounts of code, prefer creating and loading them as packages.

inline size_t countRegisteredModuleSources()

Util function for debugging which outputs the number of registered modules.

ReplicatedObj createMovable(Obj obj, InterpreterSession *I)

Converts obj from on InterpreterSession I into a ReplicatedObj.

InterpreterManager(const InterpreterManager&) = delete

InterpreterManager &operator=(const InterpreterManager&) = delete

InterpreterManager &operator=(InterpreterManager&&) = delete