Inspector APIs

Overview

The Inspector APIs provide a convenient interface for analyzing the contents of ETRecord and ETDump, helping developers get insights about model architecture and performance statistics. It’s built on top of the EventBlock Class data structure, which organizes a group of Events for easy access to details of profiling events.

There are multiple ways in which users can interact with the Inspector APIs:

• By using public methods provided by the Inspector class.

• By accessing the public attributes of the Inspector, EventBlock, and Event classes.

• By using a CLI tool for basic functionalities.

Please refer to the e2e use case doc get an understanding of how to use these in a real world example.

Inspector Methods

Constructor

sdk.Inspector.__init__(self, etdump_path=None, etrecord_path=None, source_time_scale=TimeScale.NS, target_time_scale=TimeScale.MS)

Initialize an Inspector instance with the underlying EventBlocks populated with data from the provided ETDump path and optional ETRecord path.

Parameters

• etdump_path – Path to the ETDump file.

• etrecord_path – Optional path to the ETRecord file.

• source_time_scale – The time scale of the performance data retrieved from the runtime. The default time hook implentation in the runtime returns NS.

• target_time_scale – The target time scale to which the users want their performance data converted to. Defaults to MS.

Returns

None

Example Usage:

from executorch.sdk import Inspector

inspector = Inspector(etdump_path="/path/to/etdump.etdp", etrecord_path="/path/to/etrecord.bin")

print_data_tabular

sdk.Inspector.print_data_tabular(self, include_units=True)

Displays the underlying EventBlocks in a structured tabular format, with each row representing an Event.

Parameters

include_units – Whether headers should include units (default true)

Returns

None

Example Usage:

inspector.print_data_tabular()

find_total_for_module

sdk.Inspector.find_total_for_module(self, module_name)

Returns the total average compute time of all operators within the specified module.

Parameters

module_name – Name of the module to be aggregated against.

Returns

Sum of the average compute time (in seconds) of all operators within the module with “module_name”.

Example Usage:

print(inspector.find_total_for_module("L__self___conv_layer"))

0.002

get_exported_program

sdk.Inspector.get_exported_program(self, graph=None)

Access helper for ETRecord, defaults to returning the Edge Dialect program.

Parameters

graph – Optional name of the graph to access. If None, returns the Edge Dialect program.

Returns

The ExportedProgram object of “graph”.

Example Usage:

print(inspector.get_exported_program())

ExportedProgram:
    class GraphModule(torch.nn.Module):
        def forward(self, arg0_1: f32[4, 3, 64, 64]):
            # No stacktrace found for following nodes
            _param_constant0 = self._param_constant0
            _param_constant1 = self._param_constant1

            ### ... Omit part of the program for documentation readability ... ###

Graph signature: ExportGraphSignature(parameters=[], buffers=[], user_inputs=['arg0_1'], user_outputs=['aten_tan_default'], inputs_to_parameters={}, inputs_to_buffers={}, buffers_to_mutate={}, backward_signature=None, assertion_dep_token=None)
Range constraints: {}
Equality constraints: []

Inspector Attributes

EventBlock Class

Access EventBlock instances through the event_blocks attribute of an Inspector instance, for example:

inspector.event_blocks

class sdk.inspector.inspector.EventBlock(name, events=<factory>, source_time_scale=TimeScale.NS, target_time_scale=TimeScale.MS)

An EventBlock contains a collection of events associated with a particular profiling/debugging block retrieved from the runtime. Each EventBlock represents a pattern of execution. For example, model initiation and loading lives in a single EventBlock. If there’s a control flow, each branch will be represented by a separate EventBlock.

Parameters

• name – Name of the profiling/debugging block.

• events – List of Events associated with the profiling/debugging block.

Event Class

Access Event instances through the events attribute of an EventBlock instance.

class sdk.inspector.inspector.Event(name, perf_data, op_types=<factory>, delegate_debug_identifier=None, debug_handles=None, stack_traces=<factory>, module_hierarchy=<factory>, is_delegated_op=None, delegate_backend_name=None, debug_data=<factory>, _instruction_id=None)

An Event corresponds to an operator instance with perf data retrieved from the runtime and other metadata from ETRecord.

Parameters

• name – Name of the profiling/debugging Event.

• perf_data – Performance data associated with the event retrived from the runtime (available attributes: p50, p90, avg, min and max).

• op_type – List of op types corresponding to the event.

• delegate_debug_identifier – Supplemental identifier used in combination with instruction id.

• debug_handles – Debug handles in the model graph to which this event is correlated.

• stack_trace – A dictionary mapping the name of each associated op to its stack trace.

• module_hierarchy – A dictionary mapping the name of each associated op to its module hierarchy.

• is_delegated_op – Whether or not the event was delegated.

• delegate_backend_name – Name of the backend this event was delegated to.

• debug_data – Intermediate data collected during runtime.

Example Usage:

for event_block in inspector.event_blocks:
    for event in event_block.events:
        if event.name == "Method::execute":
            print(event.perf_data.raw)

[175.748, 78.678, 70.429, 122.006, 97.495, 67.603, 70.2, 90.139, 66.344, 64.575, 134.135, 93.85, 74.593, 83.929, 75.859, 73.909, 66.461, 72.102, 84.142, 77.774, 70.038, 80.246, 59.134, 68.496, 67.496, 100.491, 81.162, 74.53, 70.709, 77.112, 59.775, 79.674, 67.54, 79.52, 66.753, 70.425, 71.703, 81.373, 72.306, 72.404, 94.497, 77.588, 79.835, 68.597, 71.237, 88.528, 71.884, 74.047, 81.513, 76.116]

CLI

Execute the following command in your terminal to display the data table. This command produces the identical table output as calling the print_data_tabular mentioned earlier:

python3 -m sdk.inspector.inspector_cli --etdump_path <path_to_etdump> --etrecord_path <path_to_etrecord>

Note that the etrecord_path argument is optional.

We plan to extend the capabilities of the CLI in the future.