Featured projects
TL;DR
- Deploying PyTorch models on microcontrollers has historically required converting to other framework formats, maintaining separate toolchains, and debugging cross-framework numerical mismatches. ExecuTorch eliminates this conversion step entirely.
- The Cortex-M backend enables PyTorch models to run directly on Arm Cortex-M CPUs with Arm CMSIS-NN acceleration. It supports all capability tiers (Scalar, DSP, and MVE/Helium) from a single workflow.
- We benchmark four of the five models from the MLPerf Tiny suite on the Alif Ensemble E8. Each model runs in 21.5 ms or less and uses under 90 KiB of RAM on a single Cortex-M55 core.
- For heterogeneous SoCs, the Cortex-M backend also provides the CPU execution path alongside Ethos-U NPU delegation.
- Minimal binary footprint. The ExecuTorch core runtime is under 20 KiB of flash, and operator selective build links only the operators your model needs, so total code size scales with the model rather than the framework.
Background
Microcontrollers power billions of always-on devices: wearables, industrial sensors, voice interfaces, and smart home appliances. These devices operate under extreme constraints: often less than 1 MB of SRAM, single-digit megabytes of flash, and power budgets measured in milliwatts. Despite these limitations, there’s growing demand to run meaningful ML workloads on these platforms and the hardware is evolving to meet it. Modern Cortex-M based microcontroller units (MCUs) now include DSP & vector ISAs, large tightly-coupled memories (TCMs), and power-efficient NPUs such as the Arm Ethos-U NPU that make ML inference more practical in resource constrained devices.
Until now, deploying ML on Cortex-M meant leaving the PyTorch ecosystem. Developers had to convert models to alternative formats, maintain separate toolchains for training and deployment, and debug numerical mismatches introduced during conversion. For teams that train in PyTorch, which accounts for over 90% of the research code published at top ML conferences, this translation step adds friction to every iteration cycle.
ExecuTorch was designed to eliminate this gap. Arm MCU support has been available through the Ethos-U NPU, and the Cortex-M backend extends it to Cortex-M CPUs, so a model can run on parts with or without an accelerator. The trained PyTorch model is exported, quantized and lowered for the device within the same stack that serves models on mobile, desktop and server.
The Cortex-M Backend
The ExecuTorch Cortex-M backend replaces standard PyTorch operators with CMSIS-NN-backed custom operators through ahead-of-time compiler passes. At inference time, these operators call directly into Arm’s CMSIS-NN library, an open source collection of optimized neural network kernels for Cortex-M processors.
Key properties:
- CMSIS-NN accelerated. Quantized operations (linear, conv, add, mul, softmax, activations, etc.) are replaced with optimized implementations that call CMSIS-NN functions and additional optimized Cortex-M implementations directly.
- Target-aware compilation. The CPU variant (M0 through M85) and ISA tier are specified at compile time, so the correct CMSIS-NN backend is selected, eliminating runtime dispatch overhead.
- Purpose-built quantizer. Produces INT8 quantized graphs matching the data layout requirements of CMSIS-NN kernels, using PyTorch’s PT2E quantization framework.
- Single compiler toolchain. The Cortex-M lowering path operates directly on the ExecuTorch edge dialect, keeping the compilation pipeline simple and debuggable.
Cortex-M Capability Tiers
The backend targets the full Cortex-M spectrum:
| Tier | Cores | Key Feature | Use Case |
|---|---|---|---|
| Scalar | M0, M0+, M3, M23 | Standard ARM instructions | Ultra-low-cost, minimal silicon |
| DSP | M4, M7, M33, M35P | SIMD multiply-accumulate | Mid-range MCUs, signal processing |
| MVE (aka Helium) | M55, M85 | 128-bit vector ops, beat-based execution | High-performance MCU ML inference |
Arm Helium technology is where the biggest performance gains are. CMSIS-NN leverages Helium’s beat-based execution model to pipeline data movement and compute, providing significant throughput gains for the depthwise and pointwise convolutions common in edge ML architectures. That said, the same workflow targets all variants. The target configuration selects the appropriate backend automatically.
ExecuTorch is Embedded Friendly
Unlike mobile or cloud environments that must accommodate varied hardware at runtime, microcontroller deployments target a fixed hardware profile. Embedded developers possess precise knowledge of their deployment target, including the exact CPU core, available ISA extensions, SRAM allocation, and flash capacity.
ExecuTorch’s ahead-of-time stack is designed to exploit this. ExecuTorch makes target-specific decisions at compile time: graph-level optimizations, memory planning, weight packing, and kernel selection are all target-aware at compile time.
Concretely, for Cortex-M this means:
- Graph-level compute elimination. Redundant dtype conversions, quantize/dequantize pairs, and memory format transposes are removed during lowering, so only necessary compute reaches the device.
- Quantization parameter folding. Scale and zero-point arithmetic is precomputed into integer multiplier/shift form at export, so quantized kernel inference stays entirely in fixed-point.
- AOT weight layout transposition. Convolution weights are transposed from PyTorch’s native layout (OIHW) to CMSIS-NN’s expected format (OHWI) at export time. The permute cost is paid once during compilation and never at inference.
- Static memory planning across heterogeneous memories. All tensor and scratch buffers are allocated and laid out ahead of time, with support for spanning multiple memory regions (e.g., TCM and SRAM). Developers can also customize placement to put latency-critical buffers in fast local memory. No heap allocator, no fragmentation, and no runtime memory management required.
- Target-specific operator replacement. The CortexMPassManager selects CMSIS-NN kernel variants matched to your core’s ISA tier (Scalar, DSP, or MVE) at compile time, not at runtime.
Developer Workflow
import torch
from torchao.quantization.pt2e.quantize_pt2e import convert_pt2e, prepare_pt2e
from executorch.backends.cortex_m.edge_compile_config import cortex_m_edge_compile_config
from executorch.backends.cortex_m.passes.cortex_m_pass_manager import CortexMPassManager
from executorch.backends.cortex_m.quantizer.quantizer import CortexMQuantizer
from executorch.backends.cortex_m.target_config import CortexM, CortexMTargetConfig
from executorch.exir import to_edge_transform_and_lower
# 1. Export
model = MyModel().eval()
example_inputs = (torch.randn(1, 1, 49, 10),)
exported = torch.export.export(model, example_inputs)
# 2. Quantize and calibrate
quantizer = CortexMQuantizer(use_explicit_layout=True)
prepared = prepare_pt2e(exported.module(), quantizer)
for sample in calibration_data:
prepared(sample)
quantized = convert_pt2e(prepared)
# 3. Re-export the quantized model
quantized_exported = torch.export.export(quantized, example_inputs)
# 4. Lower to Edge and apply the Cortex-M passes
pass_manager = CortexMPassManager(
target_config=CortexMTargetConfig(cpu=CortexM.M55),
use_explicit_layout=True,
)
edge = to_edge_transform_and_lower(
quantized_exported,
compile_config=cortex_m_edge_compile_config(),
transform_passes=pass_manager,
)
# 5. Serialize for integration into firmware
et_program = edge.to_executorch()
with open("model.pte", "wb") as f:
f.write(et_program.buffer)
The resulting .pte binary is linked into firmware alongside the ExecuTorch runtime and CMSIS-NN library, then flashed to the device. At runtime, inference executes the precompiled graph with minimal runtime overhead and no dynamic memory allocation for tensors.
Using the Cortex-M Backend Alongside Ethos-U
Many Cortex-M MCUs pair the CPU with an Arm Ethos-U NPU such as the U55, U65, or U85. ExecuTorch’s EthosUPartitioner delegates supported operations to the NPU via the Vela compiler, and operations left outside the partition run on the Cortex-M CPU within the same graph.
The Cortex-M passes run after Ethos-U partitioning and reuse the quantization parameters already in the graph. This allows a model quantized with EthosUQuantizer to use CMSIS-NN for CPU operations whose quantization, layout and shape constraints meet the Cortex-M backend requirements. Otherwise they fall back to ExecuTorch’s portable kernels, which are functional but not accelerated.
The practical benefit is that NPU operator coverage is not an all-or-nothing gate. A model with a handful of undelegated operations can ship today on mixed CPU/NPU execution, and move more of the graph onto the NPU as coverage grows.
MLPerf Tiny Benchmark Suite
We measured inference latency and memory usage across four of the five MLPerf Tiny Inference benchmark models on the Alif Ensemble E8. These benchmark models represent a set of common deep neural networks representative of TinyML workloads. The Alif E8 integrates multiple Cortex-M55 cores alongside Ethos-U55 and U85 NPUs; we measured on a single M55 high performance core running at 400 MHz, with the Ethos-U NPUs unused. ExecuTorch’s ahead-of-time memory planning means RAM usage is deterministic and bounded, with no fragmentation risk under sustained workloads. ExecuTorch’s operator selective build was used to only include those necessary for inference, creating a minimal flash footprint. All models were INT8 quantized using the CortexMQuantizer and the Alif ML Embedded Evaluation Kit was used as a test harness. These are our own measurements rather than an official MLPerf Tiny submission, so they are not directly comparable to published results on the MLCommons leaderboard.
All four models run in under 90 KiB of RAM, three of them in under 60 KiB, and the anomaly detection autoencoder in 5.2 KiB. Code footprint stays between 15 and 37 KiB because operator selective build links only the operators each model uses.
| Model | Task | Latency (ms) | Inferences per second | Peak RAM (KiB) | Flash (KiB) | ||
|---|---|---|---|---|---|---|---|
| Model | Code * | Total | |||||
| DS-CNN | Keyword Spotting | 5.825 | 172 | 26.00 | 37.67 | 36.03 | 73.70 |
| MobileNet v1 0.25 | Visual Wake Words | 21.450 | 47 | 85.15 | 253.07 | 36.03 | 289.10 |
| ResNet-8 | Image Classification | 15.290 | 65 | 55.14 | 89.77 | 29.82 | 119.59 |
| Autoencoder | Anomaly Detection | 1.208 | 828 | 5.20 | 272.03 | 14.97 | 287.00 |
* Code compiled with GCC 15.2.1 includes ExecuTorch core runtime (-Oz) and kernels (-O3) with logging disabled.
Demo: Real-time Person Detector
To show what the Cortex-M backend enables in practice, we deployed a µYOLO person detection model on a Renesas EK-RA8D1 board, powered by the Arm Cortex-M85. The deployment fits comfortably within MCU-class memory constraints:
- Task: Real-time person detection
- Model: µYOLO, pruned depthwise separable CNN backbone with linear predictor head
- Model size: 452 KiB (INT8 .pte file)
- Total flash: 640 KiB
- Peak RAM: 254 KiB
- Inference latency: 24 ms per frame inference @ 480 MHz
- End-to-end throughput: 12.7 FPS
- Input: 128×128 RGB camera frame
This example of resource-constrained, always-on vision on the edge supports applications such as robotics sensing, occupancy monitoring, safety systems, smart cameras, and presence-triggered devices.
Just as importantly, the example demonstrates the development workflow enabled by ExecuTorch. The model definition, training code, export flow, and embedded deployment live within the same PyTorch-based ecosystem. Intermediate representations remain executable, making it possible to compare model outputs throughout export and lowering rather than treating deployment as a black box. The resulting graphs can also be inspected with tools such as Model Explorer, while ETDump exposes per-operator execution costs for profiling on the target. Together, these tools make it possible to iterate on model architecture, accuracy, memory use, and latency while continuously checking that transformations have preserved model behaviour. Instead of exporting a model and only discovering numerical or performance regressions after deployment, developers can inspect and execute intermediate stages of the transformation pipeline.
The model definition, training and export flow are available in the ExecuTorch repository. To facilitate an easy bring-up of the demo without the need for a physical target, the in-tree application is targeted for the Corstone FVP. Board-specific bring-up for the on-device demo was based on code from the ra-fsp-examples repo, configured with the RA Smart Configurator, built using the Arm GNU Toolchain, and flashed using J-Link Flash. For measurements above, the weights of the most expensive operators were placed in DTCM, and the instruction cache was enabled. This resulted in an inference latency reduction from 191 ms to 24 ms, a nearly 8x improvement.
Getting Started
The Cortex-M backend is available today in the ExecuTorch repository:
| Resource | Description |
|---|---|
| Cortex-M Backend Documentation | Supported operators, target configuration, and build setup |
| Arm Backend Examples | Full example directory with setup scripts |
| CMSIS-ExecuTorch Project Template | Docker build + Keil Studio + AVH simulation |
| Zephyr RTOS Integration | ExecuTorch on Zephyr with CMSIS Toolbox |
| Raspberry Pi Pico 2 Example | Cortex-M33, MNIST digit classification |
Platforms
| Platform | CPU | Available via |
|---|---|---|
| Arm Corstone-300 FVP | Cortex-M55 + Ethos-U55 | In-tree examples, exercised in CI |
| Arm Corstone-320 FVP | Cortex-M85 + Ethos-U85 | In-tree examples, exercised in CI |
| Alif Ensemble E8 | Cortex-M55 (MVE) | Alif ML Embedded Evaluation Kit |
| Raspberry Pi Pico 2 | Cortex-M33 (DSP) | In-tree example |
| Renesas EK-RA8D1 | Cortex-M85 | Out-of-tree board port; in-tree demo runs on Corstone FVP |
What’s Next
- Expanded mixed-precision support. Extending A8W4 and A16W8 mixed-precision support across the full operator set.
- Floating-point kernels. CMSIS-NN 8.0.0 introduced fp32 and fp16 MVE kernels. Exposing them via the Cortex-M backend enables non-fully quantized models to use the same lowering path without separate deployment flows.
- Broader operator coverage. Adding support for attention operators, layer normalization, and other popular architecture patterns beyond traditional CNNs, along with CMSIS-DSP operators such as FFT so feature extraction can run as part of the graph.
- More end-to-end examples. Voice activity detection, gesture recognition, predictive maintenance, and streaming wake word, the fifth MLPerf Tiny benchmark, which needs test infrastructure for multi-frame inference.
- Additional hardware validation. Expanding the set of validated platforms and development kits.
We’re excited about what this unlocks for the embedded ML community. If you’re deploying ML at the edge, whether on a $1 sensor node or a Cortex-M55 running voice commands, ExecuTorch now provides a direct path from PyTorch training to on-device inference.
Learn more at ExecuTorch Documentation | ExecuTorch repo | Cortex-M Backend | ExecuTorch Discord