Inspector APIs

Overview

The Inspector APIs provide a convenient interface for analyzing the contents of ETRecord and ETDump, helping developers get insights about model architecture and performance statistics. It’s built on top of the EventBlock Class data structure, which organizes a group of Events for easy access to details of profiling events.

There are multiple ways in which users can interact with the Inspector APIs:

• By using public methods provided by the Inspector class.

• By accessing the public attributes of the Inspector, EventBlock, and Event classes.

• By using a CLI tool for basic functionalities.

Please refer to the e2e use case doc get an understanding of how to use these in a real world example.

Inspector Methods

Constructor

executorch.devtools.Inspector.__init__(self, etdump_path=None, etrecord=None, source_time_scale=TimeScale.NS, target_time_scale=TimeScale.MS, debug_buffer_path=None, delegate_metadata_parser=None, delegate_time_scale_converter=None, enable_module_hierarchy=False)

Initialize an Inspector instance with the underlying EventBlocks populated with data from the provided ETDump path and optional ETRecord path.

Parameters

• etdump_path – Path to the ETDump file.

• etrecord – Optional ETRecord object or path to the ETRecord file.

• source_time_scale – The time scale of the performance data retrieved from the runtime. The default time hook implentation in the runtime returns NS.

• target_time_scale – The target time scale to which the users want their performance data converted to. Defaults to MS.

• debug_buffer_path – Debug buffer file path that contains the debug data referenced by ETDump for intermediate and program outputs.

• delegate_metadata_parser – Optional function to parse delegate metadata from an Profiling Event. Expected signature of the function is: (delegate_metadata_list: List[bytes]) -> Union[List[str], Dict[str, Any]]

• delegate_time_scale_converter – Optional function to convert the time scale of delegate profiling data. If not given, use the conversion ratio of target_time_scale/source_time_scale.

• enable_module_hierarchy – Enable submodules in the operator graph. Defaults to False.

Returns

None

Example Usage:

from executorch.devtools import Inspector

inspector = Inspector(etdump_path="/path/to/etdump.etdp", etrecord="/path/to/etrecord.bin")

to_dataframe

executorch.devtools.Inspector.to_dataframe(self, include_units=True, include_delegate_debug_data=False)

Parameters

• include_units – Whether headers should include units (default true)

• include_delegate_debug_data – Whether to include delegate debug metadata (default false)

Returns

Returns a pandas DataFrame of the Events in each EventBlock in the inspector, with each row representing an Event.

print_data_tabular

executorch.devtools.Inspector.print_data_tabular(self, file=<sphinx_gallery.gen_rst._LoggingTee object>, include_units=True, include_delegate_debug_data=False)

Displays the underlying EventBlocks in a structured tabular format, with each row representing an Event.

Parameters

• file – Which IO stream to print to. Defaults to stdout. Not used if this is in an IPython environment such as a Jupyter notebook.

• include_units – Whether headers should include units (default true)

• include_delegate_debug_data – Whether to include delegate debug metadata (default false)

Returns

None

Example Usage:

inspector.print_data_tabular()

Note that the unit of delegate profiling events is “cycles”. We’re working on providing a way to set different units in the future.

find_total_for_module

executorch.devtools.Inspector.find_total_for_module(self, module_name)

Returns the total average compute time of all operators within the specified module.

Parameters

module_name – Name of the module to be aggregated against.

Returns

Sum of the average compute time (in seconds) of all operators within the module with “module_name”.

Example Usage:

print(inspector.find_total_for_module("L__self___conv_layer"))

0.002

get_exported_program

executorch.devtools.Inspector.get_exported_program(self, graph=None)

Access helper for ETRecord, defaults to returning the Edge Dialect program.

Parameters

graph – Optional name of the graph to access. If None, returns the Edge Dialect program.

Returns

The ExportedProgram object of “graph”.

Example Usage:

print(inspector.get_exported_program())

ExportedProgram:
    class GraphModule(torch.nn.Module):
        def forward(self, arg0_1: f32[4, 3, 64, 64]):
            # No stacktrace found for following nodes
            _param_constant0 = self._param_constant0
            _param_constant1 = self._param_constant1

            ### ... Omit part of the program for documentation readability ... ###

Graph signature: ExportGraphSignature(parameters=[], buffers=[], user_inputs=['arg0_1'], user_outputs=['aten_tan_default'], inputs_to_parameters={}, inputs_to_buffers={}, buffers_to_mutate={}, backward_signature=None, assertion_dep_token=None)
Range constraints: {}
Equality constraints: []

Inspector Attributes

EventBlock Class

Access EventBlock instances through the event_blocks attribute of an Inspector instance, for example:

inspector.event_blocks

class executorch.devtools.inspector.EventBlock(name, events=<factory>, source_time_scale=TimeScale.NS, target_time_scale=TimeScale.MS, bundled_input_index=None, run_output=None, reference_output=None)

An EventBlock contains a collection of events associated with a particular profiling/debugging block retrieved from the runtime. Each EventBlock represents a pattern of execution. For example, model initiation and loading lives in a single EventBlock. If there’s a control flow, each branch will be represented by a separate EventBlock.

Parameters

• name – Name of the profiling/debugging block.

• events – List of Events associated with the profiling/debugging block.

• bundled_input_idx – Index of the Bundled Input that this EventBlock corresponds to.

• run_output – Run output extracted from the encapsulated Events

Event Class

Access Event instances through the events attribute of an EventBlock instance.

class executorch.devtools.inspector.Event(name, perf_data=None, op_types=<factory>, delegate_debug_identifier=None, debug_handles=None, stack_traces=<factory>, module_hierarchy=<factory>, is_delegated_op=None, delegate_backend_name=None, _delegate_debug_metadatas=<factory>, debug_data=<factory>, _instruction_id=None, _delegate_metadata_parser=None, _delegate_time_scale_converter=None)

An Event corresponds to an operator instance with perf data retrieved from the runtime and other metadata from ETRecord.

Parameters

• name – Name of the profiling Event, empty if no profiling event.

• perf_data – Performance data associated with the event retrived from the runtime (available attributes: p10, p50, p90, avg, min and max).

• op_type – List of op types corresponding to the event.

• delegate_debug_identifier – Supplemental identifier used in combination with instruction id.

• debug_handles – Debug handles in the model graph to which this event is correlated.

• stack_trace – A dictionary mapping the name of each associated op to its stack trace.

• module_hierarchy – A dictionary mapping the name of each associated op to its module hierarchy.

• is_delegated_op – Whether or not the event was delegated.

• delegate_backend_name – Name of the backend this event was delegated to.

• _delegate_debug_metadatas – A list of raw delegate debug metadata in string, one for each profile event. Available parsed (if parser provided) as Event.delegate_debug_metadatas Available as Event.raw_delegate_debug_metadatas

• debug_data – A list containing intermediate data collected.

• _instruction_id – Instruction Identifier for Symbolication

• _delegate_metadata_parser – Optional Parser for _delegate_debug_metadatas

Example Usage:

for event_block in inspector.event_blocks:
    for event in event_block.events:
        if event.name == "Method::execute":
            print(event.perf_data.raw)

[175.748, 78.678, 70.429, 122.006, 97.495, 67.603, 70.2, 90.139, 66.344, 64.575, 134.135, 93.85, 74.593, 83.929, 75.859, 73.909, 66.461, 72.102, 84.142, 77.774, 70.038, 80.246, 59.134, 68.496, 67.496, 100.491, 81.162, 74.53, 70.709, 77.112, 59.775, 79.674, 67.54, 79.52, 66.753, 70.425, 71.703, 81.373, 72.306, 72.404, 94.497, 77.588, 79.835, 68.597, 71.237, 88.528, 71.884, 74.047, 81.513, 76.116]

CLI

Execute the following command in your terminal to display the data table. This command produces the identical table output as calling the print_data_tabular mentioned earlier:

python3 -m devtools.inspector.inspector_cli --etdump_path <path_to_etdump> --etrecord_path <path_to_etrecord>

Note that the etrecord_path argument is optional.

We plan to extend the capabilities of the CLI in the future.