⚠️ Notice: Limited Maintenance

This project is no longer actively maintained. While existing releases remain available, there are no planned updates, bug fixes, new features, or security patches. Users should be aware that vulnerabilities may not be addressed.

TorchServe on linux aarch64 - Experimental

TorchServe has been tested to be working on linux aarch64 for some of the examples.

• Tested this on Amazon Graviton 3 instance(m7g.4x.large)

Installation

Currently installation from PyPi or installing from source works

python ts_scripts/install_dependencies.py
pip install torchserve torch-model-archiver torch-workflow-archiver

Optimizations

You can also enable this optimizations for Graviton 3 to get an improved performance. More details can be found in this blog

export DNNL_DEFAULT_FPMATH_MODE=BF16
export LRU_CACHE_CAPACITY=1024

Example

This example on Text to Speech synthesis was verified to be working on Graviton 3

To Dos

• CI

• Regression tests