June 28, 2023
The Path to Achieve Ultra-Low Inference Latency With LLaMA 65B on PyTorch/XLA
Background & State of the Art
June 22, 2023
Optimized PyTorch 2.0 Inference with AWS Graviton processors
New generations of CPUs offer significant performance improvement in machine learning (ML) inference due to specialized built-in instructions. Combined with their flexibility, high speed of development, and low operating cost, these general-purpose processors offer an alternative ML inference solution to other existing hardware solutions.
June 16, 2023
🎉 PyTorch Docathon H1 2023 Wrap-up 🎉
Thank you to all who participated in our first ever PyTorch Docathon, the results have been nothing short of amazing! We want to extend our sincerest gratitude to all the participants who made this event a resounding success. Your passion, talent, and hard work have left an indelible mark on the PyTorch documentation.
June 07, 2023
Join the PyTorch Foundation: Membership Now Open
In September 2022, we welcomed PyTorch to the Linux Foundation from Meta, which formed the PyTorch Foundation with founding members AMD, Amazon Web Services (AWS), Google, Meta, Microsoft, and NVIDIA.
May 22, 2023
Out of the box acceleration and memory savings of 🤗 decoder models with PyTorch 2.0
As part of PyTorch 2.0 release, an accelerated implementation of the attention mechanism as part of the “Better Transformer” project (and known in PyTorch as Accelerated Transformers) has been added natively into PyTorch as torch.nn.functional.scaled_dot_product_attention. This implementation leverages fused kernels from FlashAttention and Memory-efficient attention, and supports both training and inference.
