Building an ExecuTorch Android Demo App

This is forked from PyTorch Android Demo App.

This guide explains how to setup ExecuTorch for Android using a demo app. The app employs a DeepLab v3 model for image segmentation tasks. Models are exported to ExecuTorch using XNNPACK FP32 backend.

What you will learn

• How to set up a build target for Android arm64-v8a

• How to build the required ExecuTorch runtime with JNI wrapper for Android

• How to build the app with required JNI library and model file

Prerequisites

• Refer to Setting up ExecuTorch to set up the repo and dev environment.

• Download and install Android Studio and SDK.

• Supported Host OS: CentOS, macOS Ventura (M1/x86_64). See below for Qualcomm HTP specific requirements.

• Qualcomm HTP Only1: To build and run on Qualcomm’s AI Engine Direct, please follow Building and Running ExecuTorch with Qualcomm AI Engine Direct Backend for hardware and software pre-requisites.

Note

This demo app and tutorial has only been validated with arm64-v8a ABI.

Build

Ahead-Of-Time

We generate the model file for the ExecuTorch runtime in Android Demo App.

XNNPACK Delegation

For delegating DeepLab v3 to XNNPACK backend, please do the following to export the model:

export FLATC_EXECUTABLE=$(realpath third-party/flatbuffers/cmake-out/flatc)
python3 -m examples.xnnpack.aot_compiler --model_name="dl3" --delegate
mkdir -p examples/demo-apps/android/ExecuTorchDemo/app/src/main/assets/
cp dl3_xnnpack_fp32.pte examples/demo-apps/android/ExecuTorchDemo/app/src/main/assets/

For more detailed tutorial of lowering to XNNPACK, please see XNNPACK backend.

Qualcomm Hexagon NPU

For delegating to Qualcomm Hexagon NPU, please follow the tutorial here.

After generating the model, copy the model to assets directory.

python -m examples.qualcomm.scripts.deeplab_v3 -b build_android -m SM8550 -s <adb_connected_device_serial>
cp deeplab_v3/dlv3_qnn.pte examples/demo-apps/android/ExecuTorchDemo/app/src/main/assets/

Runtime

We build the required ExecuTorch runtime library to run the model.

XNNPACK

1. Configure the CMake target for the library with XNNPACK backend:

export ANDROID_NDK=<path-to-android-ndk>

rm -rf cmake-out && mkdir cmake-out && cd cmake-out
cmake .. \
    -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
    -DANDROID_ABI=arm64-v8a \
    -DBUCK2=/tmp/buck2 \
    -DEXECUTORCH_BUILD_ANDROID_DEMO_APP_JNI=ON \
    -DEXECUTORCH_BUILD_XNNPACK=ON \
    -DEXECUTORCH_BUILD_FLATC=OFF \
    -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON

When we set EXECUTORCH_BUILD_XNNPACK=ON, we will build the target xnn_executor_runner_lib which in turn is linked into libexecutorchdemo via CMake.

libexecutorchdemo.so wraps up the required XNNPACK Backend runtime library from xnn_executor_runner_lib, and adds an additional JNI layer using fbjni. This is later exposed to Java app.

2. Build the libraries:

cmake --build . -j16

Qualcomm Hexagon NPU

1. Configure the CMake target for the library with Qualcomm Hexagon NPU (HTP) backend (XNNPACK also included):

export ANDROID_NDK=<path-to-android-ndk>
export QNN_SDK=<path-to-qnn-sdk>

rm -rf cmake-out && mkdir cmake-out && cd cmake-out
cmake .. \
    -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
    -DANDROID_ABI=arm64-v8a \
    -DBUCK2=/tmp/buck2 \
    -DEXECUTORCH_BUILD_ANDROID_DEMO_APP_JNI=ON \
    -DEXECUTORCH_BUILD_XNNPACK=ON \
    -DEXECUTORCH_BUILD_FLATC=OFF \
    -DEXECUTORCH_BUILD_QNN=ON \
    -DQNN_SDK_ROOT=$QNN_SDK \
    -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON

Similar to the XNNPACK library, with this setup, we compile libexecutorchdemo.so but it adds an additional static library qnn_executorch_backend which wraps up Qualcomm HTP runtime library and registers the Qualcomm HTP backend. This is later exposed to Java app.

qnn_executorch_backend is built when we turn on CMake option EXECUTORCH_BUILD_QNN. It will include the CMakeLists.txt from backends/qualcomm where we add_library(qnn_executorch_backend STATIC).

2. Build the libraries:

cmake --build . -j16

Deploying on Device via Demo App

Steps for Deploying Model via XNNPACK

mkdir -p ../examples/demo-apps/android/ExecuTorchDemo/app/src/main/jniLibs/arm64-v8a

Copy the core libraries:

cp ./examples/demo-apps/android/jni/libexecutorchdemo.so \
   ../examples/demo-apps/android/ExecuTorchDemo/app/src/main/jniLibs/arm64-v8a

This allows the Android app to load ExecuTorch runtime with XNNPACK backend as a JNI library. Later, this shared library will be loaded by NativePeer.java in Java code.

Steps for Deploying Model via Qualcomm’s AI Engine Direct

mkdir -p ../examples/demo-apps/android/ExecuTorchDemo/app/src/main/jniLibs/arm64-v8a

We need to push some additional Qualcomm HTP backend libraries to the app. Please refer to Qualcomm docs here.

cp ${QNN_SDK_ROOT}/lib/aarch64-android/libQnnHtp.so ${QNN_SDK_ROOT}/lib/aarch64-android/libQnnHtpV69Skel.so ${QNN_SDK_ROOT}/lib/aarch64-android/libQnnHtpStub.so ${QNN_SDK_ROOT}/lib/aarch64-android/libQnnSystem.so \
   ../examples/demo-apps/android/ExecuTorchDemo/app/src/main/jniLibs/arm64-v8a

Copy the core libraries:

cp ./examples/demo-apps/android/jni/libexecutorchdemo.so \
   ../examples/demo-apps/android/ExecuTorchDemo/app/src/main/jniLibs/arm64-v8a

Running the App

1. Open the project examples/demo-apps/android/ExecuTorchDemo with Android Studio.

2. Run the app (^R).

On the phone or emulator, you can try running the model:

Takeaways

Through this tutorial we’ve learnt how to build the ExecuTorch runtime library with XNNPACK (or Qualcomm HTP) backend, and expose it to JNI layer to build the Android app running segmentation model.

Reporting Issues

If you encountered any bugs or issues following this tutorial please file a bug/issue here on Github.

---

1

This section applies only if Qualcomm HTP Backend is needed in the app. Same applies to sections with titleQualcomm Hexagon NPU.