**********
TorchServe
**********
..
image:: Pytorch_logo.png
TorchServe is a performant, flexible and easy to use tool for serving PyTorch models in production.
What's going on in TorchServe?
* `High performance Llama 2 deployments with AWS Inferentia2 using TorchServe `__
* `Naver Case Study: Transition From High-Cost GPUs to Intel CPUs and oneAPI powered Software with performance `__
* `Run multiple generative AI models on GPU using Amazon SageMaker multi-model endpoints with TorchServe and save up to 75% in inference costs `__
* `Deploying your Generative AI model in only four steps with Vertex AI and PyTorch `__
* `PyTorch Model Serving on Google Cloud TPUv5 `__
* `Monitoring using Datadog `__
* `Torchserve Performance Tuning, Animated Drawings Case-Study `__
* `Walmart Search: Serving Models at a Scale on TorchServe `__
* `Scaling inference on CPU with TorchServe `__
* `TorchServe C++ backend `__
* `Grokking Intel CPU PyTorch performance from first principles: a TorchServe case study `__
* `Grokking Intel CPU PyTorch performance from first principles( Part 2): a TorchServe case study `__
* `Case Study: Amazon Ads Uses PyTorch and AWS Inferentia to Scale Models for Ads Processing `__
* `Optimize your inference jobs using dynamic batch inference with TorchServe on Amazon SageMaker `__
* `Using AI to bring children's drawings to life `__
* `Model Serving in PyTorch `__
* `Evolution of Cresta's machine learning architecture: Migration to AWS and PyTorch `__
* `Explain Like I’m 5: TorchServe `__
* `How to Serve PyTorch Models with TorchServe `__
* `How to deploy PyTorch models on Vertex AI `__
* `Quantitative Comparison of Serving Platforms `__
.. customcardstart::
.. customcarditem::
:header: TorchServe Quick Start
:card_description: Learn how to install TorchServe and serve models.
:image: https://user-images.githubusercontent.com/880376/83180095-c44cc600-a0d7-11ea-97c1-23abb4cdbe4d.jpg
:link: getting_started.html
:tags: Quick Start
.. customcarditem::
:header: Running TorchServe
:card_description: Indepth explanation of how to run TorchServe
:image: https://raw.githubusercontent.com/pytorch/serve/master/docs/images/dogs-after.jpg
:link: server.html
:tags: Running TorchServe
.. customcarditem::
:header: Why TorchServe
:card_description: Various TorchServe use cases
:image: https://download.pytorch.org/torchaudio/tutorial-assets/thumbnails/streamreader_basic_tutorial.png
:link: use_cases.html
:tags: Examples
.. customcarditem::
:header: Performance
:card_description: Guides and best practices on how to improve perfromance when working with TorchServe
:image: https://raw.githubusercontent.com/pytorch/serve/master/benchmarks/predict_latency.png
:link: performance_guide.html
:tags: Performance,Troubleshooting
.. customcarditem::
:header: Metrics
:card_description: Collecting and viewing Torcherve metrics
:image: https://user-images.githubusercontent.com/5276346/234725829-7f60e0d8-c76d-4019-ac8f-7d60069c4e58.png
:link: metrics.html
:tags: Metrics,Performance,Troubleshooting
.. customcarditem::
:header: Large Model Inference
:card_description: Serving Large Models with TorchServe
:image: https://raw.githubusercontent.com/pytorch/serve/master/docs/images/ts-lmi-internal.png
:link: large_model_inference.html
:tags: Large-Models,Performance
.. customcarditem::
:header: Troubleshooting
:card_description: Various updates on Torcherve and use cases.
:image: https://raw.githubusercontent.com/pytorch/serve/master/benchmarks/snake_viz.png
:link: Troubleshooting.html
:tags: Troubleshooting,Performance
.. customcarditem::
:header: TorchServe Security Policy
:card_description: Security Policy
:image: https://user-images.githubusercontent.com/880376/83180095-c44cc600-a0d7-11ea-97c1-23abb4cdbe4d.jpg
:link: security.html
:tags: Security
.. customcarditem::
:header: FAQs
:card_description: Various frequently asked questions.
:image: https://raw.githubusercontent.com/pytorch/serve/master/docs/images/NMTDualTranslate.png
:link: FAQs.html
:tags: FAQS
.. customcardend::