Blog

We have exciting news! PyTorch 2.4 now supports Intel® Data Center GPU Max Series and…

Introduction and Context Differentially Private Stochastic Gradient Descent (DP-SGD) is the canonical method for training machine…

In theory, Attention is All You Need. In practice, however, we also need optimized attention…

In this blog, we present an end-to-end Quantization-Aware Training (QAT) flow for large language models…

We are excited to announce the release of PyTorch® 2.4 (release note)! PyTorch 2.4 adds…

Abstract The Hopper (H100) GPU architecture, billed as the “first truly asynchronous GPU”, includes a…

Attention, as a core layer of the ubiquitous Transformer architecture, is a bottleneck for large…

This blog is courtesy of the PyTorch team at Arm. More details can be found here.…

Summary Originally PyTorch, used an eager mode where each PyTorch operation that forms the model…

Over the past year, Mixture of Experts (MoE) models have surged in popularity, fueled by…

Over the past year, we’ve added support for semi-structured (2:4) sparsity into PyTorch. With just…

Summary: With PyTorch distributed’s new asynchronous checkpointing feature, developed with feedback from IBM, we show how…

An efficient decoding Grouped-Query Attention with low-precision KV cache Introduction Generative AI has taken the…

Recently, we demonstrated how FSDP and selective activation checkpointing can be used to achieve 57% MFU…

This post was contributed by Intel AI in partnership with the PyTorch Foundation. In 2017,…

TLDR: We show promising results of up to a 1.46x speedup with <2% drop in accuracy on float32…

Speculative decoding is an optimization technique for inference that makes educated guesses about future tokens…

1.0 Summary We present an optimized Triton FP8 GEMM (General Matrix-Matrix Multiply) kernel TK-GEMM, which…