Introduction to the ExecuTorch Developer Tools

ExecuTorch has been designed with productivity as one of its core objectives and the ExecuTorch Developer Tools enable this through the comprehensive suite of tools it provides users to help them profile, debug, and visualize models that they have onboarded onto ExecuTorch.

All the components of the Developer Tools have been designed from the ground up with deep integration in both the export process and the runtime. This enables us to provide unique features such as linking back operator execution in the runtime to the line of code in the original eager model that this operator originated from.

Developer Tools Features

The ExecuTorch Developer Tools support the following features:

• BundledProgram is a utility tool for exporting the model bundled with a sample set of (representative) inputs and expected outputs, so that during runtime users can validate that the actual output is in fact the same as the expected output.

• Profiling models with operator level breakdown of performance stats

  • Linking back operator performance stats to source code and module hierarchy

  • Model loading and execution time

• Delegate Integration - Surfacing performance details from delegate backends

  • Link back delegate operator execution to the nodes they represent in the edge dialect graph (and subsequently linking back to source code and module hierarchy)

• Debugging - Intermediate outputs and output quality analysis

• Memory Allocation Insights - Visualize how memory is planned, where all the live tensors are at any point in time

• Visualization - Coming soon

Fundamental components of the Developer Tools

In order to fully understand and leverage the power of the Developer Tools in this section, the fundamental components that power the Developer Tools will be detailed.

ETRecord

ETRecord (ExecuTorch Record) is an artifact generated during the export process that stores the graphs and other metadata that is critical for the Developer Tools to be able to link back the performance/debug data sourced from the runtime to the source code of the eager model.

To draw a rough equivalence to conventional software development ETRecord can be considered as the binary built with debug symbols that is used for debugging in GNU Project debugger (gdb).

More details are available in the ETRecord documentation on how to generate and store an ETRecord.

ETDump

ETDump (ExecuTorch Dump) is the binary blob that is generated by the runtime after running a model. Similarly as above, to draw a rough equivalence to conventional software development, ETDump can be considered as the coredump of ExecuTorch, but in this case within ETDump we store all the performance and debug data that was generated by the runtime during model execution.

Note

If you only care about looking at the raw performance data without linking back to source code and other extensive features, an ETDump alone will be enough to leverage the basic features of the Developer Tools. For the full experience, it is recommended that the users also generate an ETRecord.

More details are available in the ETDump documentation on how to generate and store an ETDump from the runtime.

Inspector APIs

The Inspector Python APIs are the main user enrty point into the Developer Tools. They join the data sourced from ETDump and ETRecord to give users access to all the performance and debug data sourced from the runtime along with linkage back to eager model source code and module hierarchy in an easy to use API.

More details are available in the Inspector API documentation on how to use the Inspector APIs.