TorchServe is a flexible and easy to use tool for serving PyTorch models.
1.1. Basic Features¶
Serving Quick Start - Basic server usage tutorial
Model Archive Quick Start - Tutorial that shows you how to package a model archive file.
Installation - Installation procedures
Serving Models - Explains how to use torchserve
REST API - Specification on the API endpoint for TorchServe
Packaging Model Archive - Explains how to package model archive file, use
Logging - How to configure logging
Metrics - How to configure metrics
Metrics API - How to configure metrics API
Batch inference with TorchServe - How to create and serve a model with batch inference in TorchServe
Model Zoo - List of pre-trained model archives ready to be served for inference with TorchServe.
Examples - Many examples of how to package and deploy models and workflows with TorchServe
1.2. Advanced Features¶
Advanced configuration - Describes advanced TorchServe configurations.
Custom Service - Describes how to develop custom inference services.
Unit Tests - Housekeeping unit tests for TorchServe.
Benchmark - Use JMeter to run TorchServe through the paces and collect benchmark data.
TorchServe on Kubernetes - Demonstrates a Torchserve deployment in Kubernetes using Helm Chart.
1.3. Default Handlers¶
Image Classifier - This handler takes an image and returns the name of object in that image
Text Classifier - This handler takes a text (string) as input and returns the classification text based on the model vocabulary
Object Detector - This handler takes an image and returns list of detected classes and bounding boxes respectively
Image Segmenter- This handler takes an image and returns output shape as [CL H W], CL - number of classes, H - height and W - width