PyTorch Prototype Recipes¶

Created On: Mar 03, 2021 | Last Updated: Oct 10, 2024 | Last Verified: Nov 05, 2024

Prototype features are not available as part of binary distributions like PyPI or Conda (except maybe behind run-time flags). To test these features we would, depending on the feature, recommend building from master or using the nightly wheels that are made available on pytorch.org.

Level of commitment: We are committing to gathering high bandwidth feedback only on these features. Based on this feedback and potential further engagement between community members, we as a community will decide if we want to upgrade the level of commitment or to fail fast.

FX Graph Mode Quantization User Guide

Learn about FX Graph Mode Quantization.

FX,Quantization

FX Graph Mode Post Training Dynamic Quantization

Learn how to do post training dynamic quantization in graph mode based on torch.fx.

FX,Quantization

FX Graph Mode Post Training Static Quantization

Learn how to do post training static quantization in graph mode based on torch.fx.

FX,Quantization

Graph Mode Dynamic Quantization on BERT

Learn how to do post training dynamic quantization with graph mode quantization on BERT models.

Text,Quantization

PyTorch Numeric Suite Tutorial

Learn how to use the PyTorch Numeric Suite to support quantization debugging efforts.

Debugging,Quantization

How to Write a Quantizer for PyTorch 2 Export Quantization

Learn how to implement a Quantizer for PT2 Export Quantization

Quantization

PyTorch 2 Export Post Training Quantization

Learn how to use Post Training Quantization in PyTorch 2 Export.

Quantization

PyTorch 2 Export Quantization-Aware Training

Learn how to use Quantization-Aware-Training in PyTorch 2 Export.

Quantization

PyTorch 2 Export Quantization with X86 Backend through Inductor

Learn how to use PT2 Export Quantization with X86 Backend through Inductor.

Quantization

(prototype) Accelerating BERT with semi-structured (2:4) sparsity

Prune BERT to be 2:4 sparse and accelerate for inference.

Model-Optimiziation

Use iOS GPU in PyTorch

Learn how to run your models on iOS GPU.

Mobile

Convert MobileNetV2 to NNAPI

Learn how to prepare a computer vision model to use Android’s Neural Networks API (NNAPI).

Mobile

PyTorch Vulkan Backend User Workflow

Learn how to use the Vulkan backend on mobile GPUs.

Mobile

Tracing-based Selective Build Mobile Interpreter in Android and iOS

Learn how to optimize the mobile interpreter size with a tracing-based selective build.

Mobile

Convert Mobilenetv2 to Core ML

Learn how to prepare a computer vision model to use the PyTorch Core ML mobile backend.

Mobile

Skipping Module Parameter Initialization in PyTorch 1.10

Describes skipping parameter initialization during module construction in PyTorch 1.10, avoiding wasted computation.

Modules

Model Freezing in TorchScript

Freezing is the process of inlining Pytorch module parameters and attributes values into the TorchScript internal representation.

TorchScript

Using torch.vmap

Learn about torch.vmap, an autovectorizer for PyTorch operations.

vmap

Nested Tensor

Learn about nested tensors, the new way to batch heterogeneous-length data

NestedTensor

MaskedTensor Overview

Learn about masked tensors, the source of truth for specified and unspecified values

MaskedTensor

Masked Tensor Sparsity

Learn about how to leverage sparse layouts (e.g. COO and CSR) in MaskedTensor

MaskedTensor

Masked Tensor Advanced Semantics

Learn more about Masked Tensor's advanced semantics (reductions and comparing vs. NumPy's MaskedArray)

MaskedTensor

MaskedTensor: Simplifying Adagrad Sparse Semantics

See a showcase on how masked tensors can enable sparse semantics and provide for a cleaner dev experience

MaskedTensor

Inductor Cpp Wrapper Tutorial

Speed up your models with Inductor Cpp Wrapper

Model-Optimization

Inductor Windows CPU Tutorial

Speed up your models with Inductor On Windows CPU

Model-Optimization

Use max-autotune compilation on CPU to gain additional performance boost

Tutorial for max-autotune mode on CPU to gain additional performance boost

Model-Optimization

Flight Recorder Tutorial

Debug stuck jobs easily with Flight Recorder

Distributed, Debugging, FlightRecorder

Out-of-tree extension autoloading in Python

Learn how to improve the seamless integration of out-of-tree extension with PyTorch based on the autoloading mechanism.

Extending-PyTorch, Frontend-APIs

PyTorch Prototype Recipes¶

FX Graph Mode Quantization User Guide

FX Graph Mode Post Training Dynamic Quantization

FX Graph Mode Post Training Static Quantization

Graph Mode Dynamic Quantization on BERT

PyTorch Numeric Suite Tutorial

How to Write a Quantizer for PyTorch 2 Export Quantization

PyTorch 2 Export Post Training Quantization

PyTorch 2 Export Quantization-Aware Training

PyTorch 2 Export Quantization with X86 Backend through Inductor

(prototype) Accelerating BERT with semi-structured (2:4) sparsity

Use iOS GPU in PyTorch

Convert MobileNetV2 to NNAPI

PyTorch Vulkan Backend User Workflow

Tracing-based Selective Build Mobile Interpreter in Android and iOS

Convert Mobilenetv2 to Core ML

Skipping Module Parameter Initialization in PyTorch 1.10

Model Freezing in TorchScript

Using torch.vmap

Nested Tensor

MaskedTensor Overview

Masked Tensor Sparsity

Masked Tensor Advanced Semantics

MaskedTensor: Simplifying Adagrad Sparse Semantics

Inductor Cpp Wrapper Tutorial

Inductor Windows CPU Tutorial

Use max-autotune compilation on CPU to gain additional performance boost

Flight Recorder Tutorial

Out-of-tree extension autoloading in Python

Docs

Tutorials

Resources