Blog

Blog

Flight Recorder: A New Lens for Understanding NCCL Watchdog Timeouts

If you’ve ever trained a large AI model and had it fail with an error…

Phillip Liu, Uttam Thakore, Junjie Wang, Justin YangMarch 25, 2026

Blog

Enabling Up to 41% Faster Pre-training: MXFP8 and DeepEP for DeepSeek-V3 on B200 with TorchTitan

TL;DR In a joint effort between PyTorch and Nebius, we enabled training DeepSeek-V3 Mixture-of-Experts models…

PyTorch and Nebius (Hooman Ramezani) TeamsMarch 25, 2026

Blog

PyTorch 2.11 Release Blog

We are excited to announce the release of PyTorch® 2.11 (release notes)! The PyTorch 2.11…

PyTorch FoundationMarch 23, 2026

Blog

PyTorch 2.10+TorchAO: Powering AIPC scenarios on Intel® Core™ Ultra Series 3 processors

Overview We are excited to introduce the highlights of Intel® Core™ Ultra Series 3 processors…

Intel PyTorch and Client AI SW teamMarch 20, 2026

Blog

TorchSpec: Speculative Decoding Training at Scale

Introduction Over the past year, large language models have rapidly expanded in both scale and…

TorchSpec team, Mooncake teamMarch 19, 2026

Blog

Generalized Dot-Product Attention: Tackling Real-World Challenges in GPU Training Kernels

In this blog post, we present the kernel design of Generalized Dot-Product Attention (GDPA), a…

Jackie (Jiaqi) Xu, Chao Chen, Shuqi Yang, Markus Hoehnerbach, Xiaoyi (Leo) Liu, Ted Zadouri‡, Dev (Devashish) Shankar, Jacky Zhou, Hongtao Yu, Manman Ren, Han Xu, Chunzhi Yang†, Jade Nie†, Haoyu Zhang, Huayu Li, Michael Shu, Musharaf Sultan, Max Leung, John Bocharov, Tri Dao‡March 18, 2026

Blog

Building Voice Agents with ExecuTorch: A Cross-Platform Foundation for On-Device Audio

TL;DR Open source voice models are proliferating, but there's no unified native inference platform for…

Mergen Nachin, Manuel Candales, Mengwei Liu, Jacob Szwejbka, Young Han, Songhao Jia, Stephen Jia, Scott Roy, Alban Desmaison, Hansong Zhang from PyTorch Team at Meta; Yagil Burowski, Matt Clayton, Will Burford from LM StudioMarch 15, 2026

Blog

MXFP8 Training for MoEs: 1.3x training speedup vs BF16 for Llama4 Scout on GB200 cluster using TorchAO and TorchTitan

TL;DR We recently demonstrated a +30.2% training speedup for Llama4 Scout with equivalent convergence to…

Daniel Vega-Myhre,Matthias Reso,Vasiliy Kuznetsov,Driss Guessous,Simon Fan, Alireza Shamsoshoara, Chinmay BaikarMarch 12, 2026

Announcements Blog

KubeCon + CloudNativeCon + OpenInfra Summit + PyTorch Conference China 2026 CFP & Registration Now Open

Both the Call for Proposals and registration are now open for KubeCon + CloudNativeCon +…

PyTorch FoundationMarch 11, 2026

Blog

PyTorch at NVIDIA GTC 2026: Join Us in San Jose!

We're excited to announce that PyTorch will have a strong presence at NVIDIA GTC 2026,…

Clement Anthonioz Blanc, Chris Gottbrath, PyTorch Team at MetaMarch 9, 2026

Blog

KernelAgent: Hardware-Guided GPU Kernel Optimization via Multi-Agent Orchestration

Summary Recently, the PyTorch team released KernelAgent, an open agentic system achieving 100% correctness across…

Kaiming Cheng, Laura Wang, Jack Khuu, Mark Saroufim, Wenyuan Chi, Jiannan Wang, and Joe IsaacsonMarch 6, 2026

Blog

FlexAttention + FlashAttention-4: Fast and Flexible

TL;DR: On Hopper and Blackwell GPUs, FlexAttention now has a FlashAttention-4 backend. We added support…

Driss Guessous, Reuben Stern, Markus Hoehnerbach, Fung Xie, Ted Zadouri, Jay Shah, Tri DaoMarch 5, 2026

Blog

Deploying PyTorch Models to the Micro-Edge with ExecuTorch and Arm

The world of AI is expanding beyond the cloud, reaching devices that fit in the…

Dominica Abena Oforiwaa AmanfoMarch 5, 2026

Blog

Quantization-Aware Training in TorchAO (II)

In our previous Quantization-Aware Training (QAT) blog, we introduced the initial QAT flow in TorchAO…

Meta: Andrew Or, Lisa Jin, Scott Roy, Jerry Zhang, Mergen Nachin, Supriya Rao, Lin Xiao Unsloth: Daniel Han Axolotl: Salman MohammadiMarch 4, 2026

Announcements Blog

Kubetorch Joins the PyTorch Ecosystem Landscape: A Fast, Pythonic, Fault-Tolerant Interface into Kubernetes for ML

Kubetorch enables ML research and development on Kubernetes, across training, inference, RL, evals, data processing,…

Paul Yang, Donny GreenbergFebruary 27, 2026

Blog

Enhancing Multimodal Training and Memory Efficiency with DeepSpeed

Overview This blog walks through two crucial DeepSpeed updates: (1) a PyTorch-identical backward API that…

Masahiro Tanaka (Anyscale) and Olatunji Ruwase (Snowflake)February 24, 2026

Blog

Accelerating Autotuning in Helion with Bayesian Optimization

Introduction As introduced in a previous blog post, Helion is a high-level DSL that empowers…

Ethan Che, Oguz Ulgen, Max Balandat, Jongsok Choi, Jason AnselFebruary 24, 2026

PyTorch Foundation New Members Press Release

Announcements Blog

PyTorch Foundation Announces New Members as Agentic AI Demand Grows

Foundation welcomes Clockwork.io, Emmi AI, NIPA, Nota AI., Yasp, CommonAI CIC, Carnegie Mellon University, Monash…

PyTorch FoundationFebruary 24, 2026

Announcements Blog

PyTorchCon Europe Schedule is Live

The schedule for PyTorch Conference Europe is officially live! Join us 7-8 April in Paris…

PyTorch FoundationFebruary 23, 2026

Blog

Pyrefly Now Type Checks PyTorch

We’re excited to share that PyTorch now leverages Pyrefly to power type checking across our…

PyTorch and Pyrefly Teams at MetaFebruary 12, 2026

Flight Recorder: A New Lens for Understanding NCCL Watchdog Timeouts

Enabling Up to 41% Faster Pre-training: MXFP8 and DeepEP for DeepSeek-V3 on B200 with TorchTitan

PyTorch 2.11 Release Blog

PyTorch 2.10+TorchAO: Powering AIPC scenarios on Intel® Core™ Ultra Series 3 processors

TorchSpec: Speculative Decoding Training at Scale

Generalized Dot-Product Attention: Tackling Real-World Challenges in GPU Training Kernels

Building Voice Agents with ExecuTorch: A Cross-Platform Foundation for On-Device Audio

MXFP8 Training for MoEs: 1.3x training speedup vs BF16 for Llama4 Scout on GB200 cluster using TorchAO and TorchTitan

KubeCon + CloudNativeCon + OpenInfra Summit + PyTorch Conference China 2026 CFP & Registration Now Open

PyTorch at NVIDIA GTC 2026: Join Us in San Jose!

KernelAgent: Hardware-Guided GPU Kernel Optimization via Multi-Agent Orchestration

FlexAttention + FlashAttention-4: Fast and Flexible

Deploying PyTorch Models to the Micro-Edge with ExecuTorch and Arm

Quantization-Aware Training in TorchAO (II)

Kubetorch Joins the PyTorch Ecosystem Landscape: A Fast, Pythonic, Fault-Tolerant Interface into Kubernetes for ML

Enhancing Multimodal Training and Memory Efficiency with DeepSpeed

Accelerating Autotuning in Helion with Bayesian Optimization

PyTorch Foundation Announces New Members as Agentic AI Demand Grows

PyTorchCon Europe Schedule is Live

Pyrefly Now Type Checks PyTorch

Docs

Tutorials

Resources

Stay in touch for updates, event info, and the latest news