Blog

We are happy to announce the stable release, 1.0, for TorchRec and FBGEMM. TorchRec is the PyTorch native…

We are excited to announce the release of PyTorch® 2.5 (release note)! This release features…

The challenge of PyTorch’s lower CPU performance on Windows compared to Linux has been a…

We are pleased to announce the first-ever Chair and Vice Chair of the PyTorch Foundation’s…

Introduction As the demand for diverse hardware accelerators grows, the need for a robust and…

In this blog, we discuss the methods we used to achieve FP16 inference with popular…

We have exciting news! PyTorch 2.4 now supports Intel® Data Center GPU Max Series and…

Introduction and Context Differentially Private Stochastic Gradient Descent (DP-SGD) is the canonical method for training machine…

In theory, Attention is All You Need. In practice, however, we also need optimized attention…

In this blog, we present an end-to-end Quantization-Aware Training (QAT) flow for large language models…

We are excited to announce the release of PyTorch® 2.4 (release note)! PyTorch 2.4 adds…

Abstract The Hopper (H100) GPU architecture, billed as the “first truly asynchronous GPU”, includes a…

Attention, as a core layer of the ubiquitous Transformer architecture, is a bottleneck for large…

This blog is courtesy of the PyTorch team at Arm. More details can be found here.…

Summary Originally PyTorch, used an eager mode where each PyTorch operation that forms the model…

Over the past year, Mixture of Experts (MoE) models have surged in popularity, fueled by…

Over the past year, we’ve added support for semi-structured (2:4) sparsity into PyTorch. With just…