Dynamo / `torch.compile`¶

Torch-TensorRT provides a backend for the new torch.compile API released in PyTorch 2.0. In the following examples we describe a number of ways you can leverage this backend to accelerate inference.

Compiling ResNet using the Torch-TensorRT torch.compile Backend: Compiling a ResNet model using the Torch Compile Frontend for torch_tensorrt.compile
Compiling a Transformer using torch.compile and TensorRT: Compiling a Transformer model using torch.compile
Torch Compile Advanced Usage: Advanced usage including making a custom backend to use directly with the torch.compile API
Torch Compile Stable Diffusion: Compiling a Stable Diffusion model using torch.compile
Torch Export with Cudagraphs: Using the Cudagraphs integration with ir=”dynamo”
Using Custom Kernels within TensorRT Engines with Torch-TensorRT: Creating a plugin to use a custom kernel inside TensorRT engines
Refitting Torch-TensorRT Programs with New Weights: Refitting a compiled TensorRT Graph Module with updated weights
Mutable Torch TensorRT Module: Compile, use, and modify TensorRT Graph Module with MutableTorchTensorRTModule
vgg16_fp8_ptq: Compiling a VGG16 model with FP8 and PTQ using torch.compile
Engine Caching: Utilizing engine caching to speed up compilation times
Engine Caching (BERT): Demonstrating engine caching on BERT