1. TorchServe

TorchServe is a performant, flexible and easy to use tool for serving PyTorch eager mode and torschripted models.

1.1. Basic Features

1.2. Default Handlers

  • Image Classifier - This handler takes an image and returns the name of object in that image

  • Text Classifier - This handler takes a text (string) as input and returns the classification text based on the model vocabulary

  • Object Detector - This handler takes an image and returns list of detected classes and bounding boxes respectively

  • Image Segmenter- This handler takes an image and returns output shape as [CL H W], CL - number of classes, H - height and W - width

1.3. Examples

  • HuggingFace Language Model - This handler takes an input sentence and can return sequence classifications, token classifications or Q&A answers

  • Multi Modal Framework - Build and deploy a classifier that combines text, audio and video input data

  • Dual Translation Workflow -

  • Model Zoo - List of pre-trained model archives ready to be served for inference with TorchServe.

  • Examples - Many examples of how to package and deploy models with TorchServe

  • Workflow Examples - Examples of how to compose models in a workflow with TorchServe

1.4. Advanced Features


Access comprehensive developer documentation for PyTorch

View Docs


Get in-depth tutorials for beginners and advanced developers

View Tutorials


Find development resources and get your questions answered

View Resources