# ignite.contrib.metrics#

Contribution module of metrics

class ignite.contrib.metrics.AveragePrecision(activation=None, output_transform=<function AveragePrecision.<lambda>>)[source]#

Computes Average Precision accumulating predictions and the ground-truth during an epoch and applying sklearn.metrics.average_precision_score

Parameters
• activation (Callable, optional) – optional function to apply on prediction tensors, e.g. activation=torch.sigmoid to transform logits.

• output_transform (callable, optional) – a callable that is used to transform the ignite.engine.Engine’s process_function’s output into the form expected by the metric. This can be useful if, for example, you have a multi-output model and you want to compute the metric with respect to one of the outputs.

class ignite.contrib.metrics.ROC_AUC(activation=None, output_transform=<function ROC_AUC.<lambda>>)[source]#

Computes Area Under the Receiver Operating Characteristic Curve (ROC AUC) accumulating predictions and the ground-truth during an epoch and applying sklearn.metrics.roc_auc_score

Parameters
• activation (Callable, optional) – optional function to apply on prediction tensors, e.g. activation=torch.sigmoid to transform logits.

• output_transform (callable, optional) – a callable that is used to transform the ignite.engine.Engine’s process_function’s output into the form expected by the metric. This can be useful if, for example, you have a multi-output model and you want to compute the metric with respect to one of the outputs.

## Regression metrics#

Module ignite.contrib.metrics.regression provides implementations of metrics useful for regression tasks. Definitions of metrics are based on Botchkarev 2018, page 30 “Appendix 2. Metrics mathematical definitions”.

class ignite.contrib.metrics.regression.FractionalBias(output_transform=<function Metric.<lambda>>)[source]#

Calculates the Fractional Bias:

$\text{FB} = \frac{1}{n}\sum_{j=1}^n\frac{2 * (A_j - P_j)}{A_j + P_j}$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape.

compute()[source]#

Computes the metric based on it’s accumulated state.

This is called at the end of each epoch.

Returns

the actual quantity of interest

Return type

Any

Raises

NotComputableError – raised when the metric cannot be computed

reset()[source]#

Resets the metric to to it’s initial state.

This is called at the start of each epoch.

update(output)[source]#

Updates the metric’s state using the passed batch output.

This is called once for each batch.

Parameters

output – the is the output from the engine’s process function

class ignite.contrib.metrics.regression.ManhattanDistance(output_transform=<function Metric.<lambda>>)[source]#

Calculates the Manhattan Distance:

$\text{MD} = \sum_{j=1}^n (A_j - P_j)$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape.

compute()[source]#

Computes the metric based on it’s accumulated state.

This is called at the end of each epoch.

Returns

the actual quantity of interest

Return type

Any

Raises

NotComputableError – raised when the metric cannot be computed

reset()[source]#

Resets the metric to to it’s initial state.

This is called at the start of each epoch.

update(output)[source]#

Updates the metric’s state using the passed batch output.

This is called once for each batch.

Parameters

output – the is the output from the engine’s process function

class ignite.contrib.metrics.regression.MaximumAbsoluteError(output_transform=<function Metric.<lambda>>)[source]#

Calculates the Maximum Absolute Error:

$\text{MaxAE} = \max_{j=1,n} \left( \lvert A_j-P_j \rvert \right)$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape.

compute()[source]#

Computes the metric based on it’s accumulated state.

This is called at the end of each epoch.

Returns

the actual quantity of interest

Return type

Any

Raises

NotComputableError – raised when the metric cannot be computed

reset()[source]#

Resets the metric to to it’s initial state.

This is called at the start of each epoch.

update(output)[source]#

Updates the metric’s state using the passed batch output.

This is called once for each batch.

Parameters

output – the is the output from the engine’s process function

class ignite.contrib.metrics.regression.MeanAbsoluteRelativeError(output_transform=<function Metric.<lambda>>)[source]#

Calculate Mean Absolute Relative Error:

$\text{MARE} = \frac{1}{n}\sum_{j=1}^n\frac{\left|A_j-P_j\right|}{\left|A_j\right|}$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in the reference Botchkarev 2018.

• update must receive output of the form (y_pred, y)

compute()[source]#

Computes the metric based on it’s accumulated state.

This is called at the end of each epoch.

Returns

the actual quantity of interest

Return type

Any

Raises

NotComputableError – raised when the metric cannot be computed

reset()[source]#

Resets the metric to to it’s initial state.

This is called at the start of each epoch.

update(output)[source]#

Updates the metric’s state using the passed batch output.

This is called once for each batch.

Parameters

output – the is the output from the engine’s process function

class ignite.contrib.metrics.regression.MeanError(output_transform=<function Metric.<lambda>>)[source]#

Calculates the Mean Error:

$\text{ME} = \frac{1}{n}\sum_{j=1}^n (A_j - P_j)$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in the reference Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape.

compute()[source]#

Computes the metric based on it’s accumulated state.

This is called at the end of each epoch.

Returns

the actual quantity of interest

Return type

Any

Raises

NotComputableError – raised when the metric cannot be computed

reset()[source]#

Resets the metric to to it’s initial state.

This is called at the start of each epoch.

update(output)[source]#

Updates the metric’s state using the passed batch output.

This is called once for each batch.

Parameters

output – the is the output from the engine’s process function

class ignite.contrib.metrics.regression.MeanNormalizedBias(output_transform=<function Metric.<lambda>>)[source]#

Calculates the Mean Normalized Bias:

$\text{MNB} = \frac{1}{n}\sum_{j=1}^n\frac{A_j - P_j}{A_j}$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in the reference Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape.

compute()[source]#

Computes the metric based on it’s accumulated state.

This is called at the end of each epoch.

Returns

the actual quantity of interest

Return type

Any

Raises

NotComputableError – raised when the metric cannot be computed

reset()[source]#

Resets the metric to to it’s initial state.

This is called at the start of each epoch.

update(output)[source]#

Updates the metric’s state using the passed batch output.

This is called once for each batch.

Parameters

output – the is the output from the engine’s process function