Blog

PyTorch DDP has been widely adopted across the industry for distributed training, which by default runs…

Congratulations to the PyTorch Foundation for its release of PyTorch 2.0! In this blog, I discuss…

As we celebrate the release of OpenXLA, PyTorch 2.0, and PyTorch/XLA 2.0, it’s worth taking a step back…

The PyTorch 2.0 release includes a new high-performance implementation of the PyTorch Transformer API with…

Today, we are excited to share our latest work for PyTorch/XLA 2.0. The release of PyTorch 2.0 is…

PyTorch 2.0 has just been released. Its flagship new feature is torch.compile(), a one-line code change…

We are excited to announce the release of PyTorch® 2.0 which we highlighted during the PyTorch Conference on 12/2/22!…

Summary We are bringing a number of improvements to the current PyTorch libraries, alongside the PyTorch…

Last year, Meta announced that PyTorch joined the Linux Foundation as a neutral home for growing the…

For the upcoming PyTorch 2.0 feature release (target March 2023), we will target CUDA 11.7…

We are excited to announce the public release of Holistic Trace Analysis (HTA), an open…

If you installed PyTorch-nightly on Linux via pip between December 25, 2022 and December 30,…

In this post we discuss performance tuning of Torchserve for serving your models in production.…

TL;DR: We demonstrate the use of PyTorch with FairScale’s FullyShardedDataParallel (FSDP) API in writing large…

Cutting-edge AI models are becoming extremely large. The cost and overhead of training these models…

Large model training using a cloud native approach is of growing interest for many enterprises…