Training MoEs at Scale with PyTorch
Over the past year, Mixture of Experts (MoE) models have surged in popularity, fueled by…
🎉 PyTorch Docathon H1 2024 Wrap-up 🎉
We are thrilled to announce the successful completion of the H1 2024 PyTorch Docathon! The…
PyTorch FoundationJune 20, 2024
Accelerating Neural Network Training with Semi-Structured (2:4) Sparsity
Over the past year, we’ve added support for semi-structured (2:4) sparsity into PyTorch. With just…
Jesse Cai, Daniel Haziza, Supriya RaoJune 20, 2024
Reducing Model Checkpointing Times by Over 10x with PyTorch Distributed Asynchronous Checkpointing
Summary: With PyTorch distributed’s new asynchronous checkpointing feature, developed with feedback from IBM, we show how…
PyTorch Foundation Welcomes New Executive Director
The PyTorch Foundation is excited to welcome Matt White, our new executive director. The PyTorch…
PyTorch FoundationJune 11, 2024
INT4 Decoding GQA CUDA Optimizations for LLM Inference
An efficient decoding Grouped-Query Attention with low-precision KV cache Introduction Generative AI has taken the…
Ready, Set, Contribute: PyTorch Docathon Kickoff H1 2024
The PyTorch Docathon is now live! This event is dedicated to enhancing the quality of…
PyTorch FoundationJune 4, 2024
AI Helps Duolingo Personalize Language Learning
![]()
Learning a foreign language was probably one of your goals last year. And the year…
PyTorch FoundationMay 25, 2024
Maximizing Training Throughput Using PyTorch FSDP and Torch.compile
Recently, we demonstrated how FSDP and selective activation checkpointing can be used to achieve 57% MFU…
Team PyTorch at IBM and Team PyTorch at MetaMay 21, 2024
Achieving Sustainability Goals with PyTorch and Intel AI
This post was contributed by Intel AI in partnership with the PyTorch Foundation. In 2017,…
PyTorch FoundationMay 15, 2024
Speeding up ViTs using Block Sparsity
TLDR: We show promising results of up to a 1.46x speedup with <2% drop in accuracy on float32…
Introducing depyf: mastering torch.compile with ease
We are thrilled to introduce depyf, a new project to the PyTorch ecosystem designed to help…
Kaichao YouMay 11, 2024
Deep Learning Energy Measurement and Optimization
This post is authored by Jae-Won Chung, a PhD student at the University of Michigan and…
Jae-Won ChungMay 11, 2024
Enhancing Deep Learning Workflows: PyTorch Ecosystem Tools
Welcome to the thriving PyTorch ecosystem, where a wealth of tools and libraries await, purpose-built…
PyTorch FoundationMay 11, 2024
A Hitchhiker’s Guide to Speculative Decoding
Speculative decoding is an optimization technique for inference that makes educated guesses about future tokens…
Team PyTorch at IBMMay 2, 2024
Announcing PyTorch Docathon June, 2024
We are thrilled to announce the upcoming PyTorch Docathon in June! The Docathon, akin to…
PyTorch FoundationMay 2, 2024
Accelerating Llama3 FP8 Inference with Triton Kernels
1.0 Summary We present an optimized Triton FP8 GEMM (General Matrix-Matrix Multiply) kernel TK-GEMM, which…
Adnan Hoque, Less Wright, Chih Chieh YangMay 1, 2024
ExecuTorch Alpha: Taking LLMs and AI to the Edge with Our Community and Partners
We are excited to announce the release of ExecuTorch alpha, focused on deploying large language models…
PyTorch FoundationApril 30, 2024
PyTorch 2.3 Release Blog
We are excited to announce the release of PyTorch® 2.3 (release note)! PyTorch 2.3 offers…
PyTorch FoundationApril 24, 2024
torchtune: Easily fine-tune LLMs using PyTorch
We’re pleased to announce the alpha release of torchtune, a PyTorch-native library for easily fine-tuning…
PyTorch FoundationApril 16, 2024