Category

Blog

Supercharging LLMs: Scalable RL with torchforge and Weaver Blog

Supercharging LLMs: Scalable RL with torchforge and Weaver

Scaling reinforcement learning (RL) for post-training large language models (LLMs) is notoriously difficult. While running…
Stanford - Jon Saad-Falcon, Hangoo Kang, Simon Guo, Aakanksha Chowdhery, Azalia Mirhoseini Meta - Allen Wang, Danning Xie, Evan Smothers, Felipe Mello, Jack Khuu, Jiyue Wang, Joe Cummings, Lucas Pasqualin, Philip Bontrager, Rithesh Baradi, Vidhya Venkat, Yuxuan Hu, Jafar Taghiyar, Davide Italiano, Gayathri Aiyer, John Myles White, Joe Spisak, Sanyam Bhutani, Hamid Shojanazeri, Matthias Reso Ali Sol Hossein Kavianihamedani Emre Guven CoreWeave - Deok Filho Aaron Batilo Matthew Guan Xi Lu
January 9, 2026
Efficient MoE Pre-training at Scale on 1K AMD GPUs with TorchTitan Blog

Efficient MoE Pre-training at Scale on 1K AMD GPUs with TorchTitan

Training massive Mixture-of-Experts (MoE) models like DeepSeek-V3 and Llama 4-Scout efficiently is one of the…
AMD Contributors: Liz Li, Yanyuan Qin, Yuankai Chen, Xinyu Kang, Xiaobo Chen, Zhen Huang, Shekhar Pandey, Zhenyu Gu, Andy Luo, Meta Contributors: Matthias Reso, Hamid Shojanazeri, Tianyu Liu, Jiani Wang, Howard Huang, Wei Feng, Special Thanks: Guru MP, Yao Fu, Nick Ni, Emad Barsoum, Ramine Roane, and the TensorWave team for providing MI325 cluster
December 1, 2025