Torch Distributed Experimental

Torch Distributed Experimental, or in short torchdistX, contains a collection of experimental features for which our team wants to gather feedback from our users before introducing them in the core PyTorch Distributed package. In a sense features included in torchdistX can be considered in an incubation period.

Note

Please be advised that all features in torchdistX are subject to change and, although our team will make its best effort, we do not guarantee any API or ABI compatibility between releases. This means you should exercise caution if you plan to use torchdistX in production.

Installation

Check out this section in our README for installation instructions.

Documentation

Features

• Fake Tensor
• Deferred Module Initialization

Design Notes

• Fake Tensors & Deferred Module Initialization