torch::deploy [Beta]

torch::deploy is a system that allows you to load multiple python interpreters which execute PyTorch models, and run them in a single C++ process. Effectively, it allows people to multithread their pytorch models. For more information on how torch::deploy works please see the related arXiv paper. We plan to further generalize torch::deploy into a more generic system, multipy::runtime, which is more suitable for arbitrary python programs rather than just pytorch applications.

Documentation

Usage

• Installation
  • Building torch::deploy via Docker
  • Installing via pip install
  • Running torch::deploy build steps from source
  • Running unit tests for torch::deploy
• torch::deploy tutorials
  • Quickstart
  • Hello World (Using InterpreterSession directly)
  • Passing Python Objects from one Interpreter to Another
• Library API
  • Class Hierarchy
  • File Hierarchy
  • Full API

Acknowledgements

This documentation website for the MultiPy C++ API has been enabled by the Exhale project and generous investment of time and effort by its maintainer, svenevs. We thank Stephen for his work and his efforts providing help with both the PyTorch and MultiPy C++ documentation.