Shortcuts

Torch-TensorRT

In-framework compilation of PyTorch inference code for NVIDIA GPUs

Torch-TensorRT is a inference compiler for PyTorch, targeting NVIDIA GPUs via NVIDIA’s TensorRT Deep Learning Optimizer and Runtime. It supports both just-in-time (JIT) compilation workflows via the torch.compile interface as well as ahead-of-time (AOT) workflows. Torch-TensorRT integrates seamlessly into the PyTorch ecosystem supporting hybrid execution of optimized TensorRT code with standard PyTorch code.

More Information / System Architecture:

Getting Started

User Guide

Tutorials

Dynamo Frontend

TorchScript Frontend

FX Frontend

Model Zoo

Python API Documentation

C++ API Documentation

CLI Documentation

Contributor Documentation

Indices

Legacy Further Information (TorchScript)

Docs

Access comprehensive developer documentation for PyTorch

View Docs

Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

Resources

Find development resources and get your questions answered

View Resources