Inspector APIs¶
Overview¶
The Inspector APIs provide a convenient interface for analyzing the contents of ETRecord and ETDump, helping developers get insights about model architecture and performance statistics. It’s built on top of the EventBlock Class data structure, which organizes a group of Events for easy access to details of profiling events.
There are multiple ways in which users can interact with the Inspector APIs:
By using public methods provided by the
Inspector
class.By accessing the public attributes of the
Inspector
,EventBlock
, andEvent
classes.By using a CLI tool for basic functionalities.
Please refer to the e2e use case doc get an understanding of how to use these in a real world example.
Inspector Methods¶
Constructor¶
- sdk.Inspector.__init__(self, etdump_path=None, etrecord_path=None, source_time_scale=TimeScale.NS, target_time_scale=TimeScale.MS)¶
Initialize an Inspector instance with the underlying EventBlocks populated with data from the provided ETDump path and optional ETRecord path.
- Parameters
etdump_path – Path to the ETDump file.
etrecord_path – Optional path to the ETRecord file.
source_time_scale – The time scale of the performance data retrieved from the runtime. The default time hook implentation in the runtime returns NS.
target_time_scale – The target time scale to which the users want their performance data converted to. Defaults to MS.
- Returns
None
Example Usage:
from executorch.sdk import Inspector
inspector = Inspector(etdump_path="/path/to/etdump.etdp", etrecord_path="/path/to/etrecord.bin")
print_data_tabular¶
- sdk.Inspector.print_data_tabular(self, include_units=True)¶
Displays the underlying EventBlocks in a structured tabular format, with each row representing an Event.
- Parameters
include_units – Whether headers should include units (default true)
- Returns
None
Example Usage:
inspector.print_data_tabular()
find_total_for_module¶
- sdk.Inspector.find_total_for_module(self, module_name)¶
Returns the total average compute time of all operators within the specified module.
- Parameters
module_name – Name of the module to be aggregated against.
- Returns
Sum of the average compute time (in seconds) of all operators within the module with “module_name”.
Example Usage:
print(inspector.find_total_for_module("L__self___conv_layer"))
0.002
get_exported_program¶
- sdk.Inspector.get_exported_program(self, graph=None)¶
Access helper for ETRecord, defaults to returning the Edge Dialect program.
- Parameters
graph – Optional name of the graph to access. If None, returns the Edge Dialect program.
- Returns
The ExportedProgram object of “graph”.
Example Usage:
print(inspector.get_exported_program())
ExportedProgram:
class GraphModule(torch.nn.Module):
def forward(self, arg0_1: f32[4, 3, 64, 64]):
# No stacktrace found for following nodes
_param_constant0 = self._param_constant0
_param_constant1 = self._param_constant1
### ... Omit part of the program for documentation readability ... ###
Graph signature: ExportGraphSignature(parameters=[], buffers=[], user_inputs=['arg0_1'], user_outputs=['aten_tan_default'], inputs_to_parameters={}, inputs_to_buffers={}, buffers_to_mutate={}, backward_signature=None, assertion_dep_token=None)
Range constraints: {}
Equality constraints: []
Inspector Attributes¶
EventBlock
Class¶
Access EventBlock
instances through the event_blocks
attribute
of an Inspector
instance, for example:
inspector.event_blocks
- class sdk.inspector.inspector.EventBlock(name, events=<factory>, source_time_scale=TimeScale.NS, target_time_scale=TimeScale.MS)[source]¶
An EventBlock contains a collection of events associated with a particular profiling/debugging block retrieved from the runtime. Each EventBlock represents a pattern of execution. For example, model initiation and loading lives in a single EventBlock. If there’s a control flow, each branch will be represented by a separate EventBlock.
- Parameters
name – Name of the profiling/debugging block.
events – List of Events associated with the profiling/debugging block.
Event
Class¶
Access Event
instances through the events
attribute of an
EventBlock
instance.
- class sdk.inspector.inspector.Event(name, perf_data, op_types=<factory>, delegate_debug_identifier=None, debug_handles=None, stack_traces=<factory>, module_hierarchy=<factory>, is_delegated_op=None, delegate_backend_name=None, debug_data=<factory>, _instruction_id=None)[source]¶
An Event corresponds to an operator instance with perf data retrieved from the runtime and other metadata from ETRecord.
- Parameters
name – Name of the profiling/debugging Event.
perf_data – Performance data associated with the event retrived from the runtime (available attributes: p50, p90, avg, min and max).
op_type – List of op types corresponding to the event.
delegate_debug_identifier – Supplemental identifier used in combination with instruction id.
debug_handles – Debug handles in the model graph to which this event is correlated.
stack_trace – A dictionary mapping the name of each associated op to its stack trace.
module_hierarchy – A dictionary mapping the name of each associated op to its module hierarchy.
is_delegated_op – Whether or not the event was delegated.
delegate_backend_name – Name of the backend this event was delegated to.
debug_data – Intermediate data collected during runtime.
Example Usage:
for event_block in inspector.event_blocks:
for event in event_block.events:
if event.name == "Method::execute":
print(event.perf_data.raw)
[175.748, 78.678, 70.429, 122.006, 97.495, 67.603, 70.2, 90.139, 66.344, 64.575, 134.135, 93.85, 74.593, 83.929, 75.859, 73.909, 66.461, 72.102, 84.142, 77.774, 70.038, 80.246, 59.134, 68.496, 67.496, 100.491, 81.162, 74.53, 70.709, 77.112, 59.775, 79.674, 67.54, 79.52, 66.753, 70.425, 71.703, 81.373, 72.306, 72.404, 94.497, 77.588, 79.835, 68.597, 71.237, 88.528, 71.884, 74.047, 81.513, 76.116]
CLI¶
Execute the following command in your terminal to display the data table. This command produces the identical table output as calling the print_data_tabular mentioned earlier:
python3 -m sdk.inspector.inspector_cli --etdump_path <path_to_etdump> --etrecord_path <path_to_etrecord>
Note that the etrecord_path argument is optional.
We plan to extend the capabilities of the CLI in the future.