Blog

Overview In PyTorch 2.5.0 release, we introduced FlexAttention torch.nn.attention.flex_attention for ML researchers who’d like to…

Meta: Less Wright, Meet Vadakkanchery, Saurabh Mishra, Ela Krepska, Hamid Shojanazeri, Pradeep Fernando Crusoe: Ethan…

Meta: Less Wright, Hamid Shojanazeri, Vasiliy Kuznetsov, Daniel Vega-Myhre, Gokul Nadathur, Will Constable, Tianyu Liu,…

PyTorch 2.7 continues to deliver significant functionality and performance enhancements on Intel® GPU architectures to streamline…

We are excited to announce the release of PyTorch® 2.7 (release notes)! This release features:…

Automatic speech recognition (ASR) has revolutionized how we interact with technology, clearing the way for…

At Meta, recommendation systems are the cornerstone of delivering relevant and personalized ads to billions…

LinkedIn: Shivam Sahni, Byron Hsu, Yanning ChenMeta: Ankith Gunapal, Evan Smothers This blog explores the…

As models scale in depth, batch size, and sequence length, etc, activation memory becomes an…

This post is a follow-up to our first entry in the multi-series blog focused on how…

PyTorch* 2.6 has just been released with a set of exciting new features including torch.compile compatibility…

Meta: Hongtao Yu, Manman Ren, Bert Maher, Shane NayNVIDIA: Gustav Zhu, Shuhao Jiang Over the…

We are excited to announce the release of PyTorch® 2.6 (release notes)! This release features…

2024 has been a year of incredible growth for PyTorch. As that continues in 2025,…

Large Language Models (LLMs) are typically very resource-intensive, requiring significant amounts of memory, compute and…