Blog

Blog

Automated trace collection and analysis

In this blog, we share how we enabled the collection and analysis of PyTorch Profiler…

Anupam Bhatnagar, Brian CoutinhoSeptember 5, 2023

Blog

PyTorch/XLA SPMD: Scale Up Model Training and Serving with Automatic Parallelization

Today, we are delighted to announce PyTorch/XLA SPMD: the integration of GSPMD into PyTorch with an easy…

Yeounoh Chung, Jon Bolin, Milad Mohammadi, Jiewen Tan, Jack Cao, Joe Spisak, Alex Spiridonov, Shauheen Zahirazami, Steven Krawczyk, Wonjoo Lee Mohit Khatwani, Wanchao Liang, Vaibhav SinghAugust 31, 2023

Blog

Large Scale Training of Hugging Face Transformers on TPUs With PyTorch/XLA FSDP

AI is transforming many industries through advanced capabilities such as understanding and generating language, answering…

Alex Wertheim, Milad Mohammadi, Jack Cao, Alex Spiridonov, Joe Spisak, Lysandre Debut, Sylvain Gugger, Sourab MangrulkarAugust 24, 2023

Announcements

Intel Joins the PyTorch Foundation as a Premier Member

The PyTorch Foundation, a neutral home for the deep learning community to collaborate on the…

PyTorch FoundationAugust 10, 2023

Blog

INT8 Quantization for x86 CPU in PyTorch

Overview INT8 quantization is a powerful technique for speeding up deep learning inference on x86…

IntelAugust 7, 2023

Announcements

Hugging Face Joins the PyTorch Foundation as a Premier Member

The PyTorch Foundation, a neutral home for the deep learning community to collaborate on the…

PyTorch FoundationAugust 3, 2023

Announcements

AMD’s Journey to Openness and Performance

AMD has gained progress in building a robust software stack that supports an open ecosystem…

PyTorch FoundationAugust 1, 2023

Announcements

Performant Distributed checkpointing in Production with IBM

Last year, IBM Research began collaborating with us to onboard Fully Sharded Data Parallelism (FSDP)…

Meta: Iris Zhang, Less Wright, Rodrigo Kumpera, Chien-Chin Huang, IBM: Davis Wertheimer, Supriyo Chakraboty, Sophia Wen, Raghu Ganti, Mudhakar Srivatsa, Seethrami SeelamJuly 31, 2023

Announcements

IBM Joins the PyTorch Foundation as a Premier Member

The PyTorch Foundation, part of The Linux Foundation, is pleased to announce that IBM has…

PyTorch FoundationJuly 27, 2023

Blog

Announcing CPP-based S3 IO DataPipes

Training large deep learning models requires large datasets. Amazon Simple Storage Service (Amazon S3) is a scalable…

John He, Khaled ElGalaind, Roshani Nagmote, Daiming YangJuly 25, 2023

Blog

How to Accelerate PyTorch Geometric on Intel® CPUs

Overview The Intel PyTorch team has been collaborating with the PyTorch Geometric (PyG) community to…

IntelJuly 10, 2023

Community

Unveiling the Power of Semi-Supervised Learning: The Unified Semi-Supervised Learning Benchmark

Machine Learning models thrive on high-quality, fully-annotated data. The traditional supervised learning approach typically requires…

Jindong WangJuly 6, 2023

Blog

Optimizing LibTorch-based inference engine memory usage and thread-pooling

Outline In this blog post we show how to optimize LibTorch-based inference engine to maximize…

Himalay Mohanlal Joriwal, Pierre-Yves Aquilanti, Vivek Govindan, Hamid Shojanazeri, Ankith Gunapal, Tristan RiceJune 29, 2023

Community

Introducing TorchOpt: A High-Performance Differentiable Optimization Library for PyTorch

Explore TorchOpt, a PyTorch-based library that revolutionizes differentiable optimization with its unified programming abstraction, high-performance…

Benjamin LiuJune 29, 2023

Blog

The Path to Achieve Ultra-Low Inference Latency With LLaMA 65B on PyTorch/XLA

Background & State of the Art In the natural language processing (NLP) space, language models…

Milad Mohammadi, Jiewen Tan, Liyang Lu, Siyuan Liu, Yeounoh Chung, Wonjoo Lee, Manfei Bai, Steven Krawczyk, Shauheen Zahirazami, Alex Wertheim, Meghan Cowan, Jack Cao, Joe SpisakJune 28, 2023

Blog

Optimized PyTorch 2.0 Inference with AWS Graviton processors

New generations of CPUs offer significant performance improvement in machine learning (ML) inference due to…

Sunita Nadampalli from AWS & Ankith Gunapal from MetaJune 22, 2023

Blog

🎉 PyTorch Docathon H1 2023 Wrap-up 🎉

Thank you to all who participated in our first ever PyTorch Docathon, the results have…

PyTorch FoundationJune 16, 2023

Announcements

Join the PyTorch Foundation: Membership Now Open

In September 2022, we welcomed PyTorch to the Linux Foundation from Meta, which formed the…

PyTorch FoundationJune 7, 2023

Blog

Out of the box acceleration and memory savings of 🤗 decoder models with PyTorch 2.0

As part of PyTorch 2.0 release, an accelerated implementation of the attention mechanism as part…

Felix Marty, Younes Belkada, Hamid Shojanazeri, Driss GuessousMay 22, 2023

Announcements

PyTorch Conference 2023: Join us in San Francisco October 16-17

We’re thrilled to announce the upcoming PyTorch Conference 2023! On October 16-17, the conference will showcase…

PyTorch FoundationMay 16, 2023

Automated trace collection and analysis

PyTorch/XLA SPMD: Scale Up Model Training and Serving with Automatic Parallelization

Large Scale Training of Hugging Face Transformers on TPUs With PyTorch/XLA FSDP

Intel Joins the PyTorch Foundation as a Premier Member

INT8 Quantization for x86 CPU in PyTorch

Hugging Face Joins the PyTorch Foundation as a Premier Member

AMD’s Journey to Openness and Performance

Performant Distributed checkpointing in Production with IBM

IBM Joins the PyTorch Foundation as a Premier Member

Announcing CPP-based S3 IO DataPipes

How to Accelerate PyTorch Geometric on Intel® CPUs

Unveiling the Power of Semi-Supervised Learning: The Unified Semi-Supervised Learning Benchmark

Optimizing LibTorch-based inference engine memory usage and thread-pooling

Introducing TorchOpt: A High-Performance Differentiable Optimization Library for PyTorch

The Path to Achieve Ultra-Low Inference Latency With LLaMA 65B on PyTorch/XLA

Optimized PyTorch 2.0 Inference with AWS Graviton processors

🎉 PyTorch Docathon H1 2023 Wrap-up 🎉

Join the PyTorch Foundation: Membership Now Open

Out of the box acceleration and memory savings of 🤗 decoder models with PyTorch 2.0

PyTorch Conference 2023: Join us in San Francisco October 16-17

Docs

Tutorials

Resources

Stay in touch for updates, event info, and the latest news