# ignite.contrib.metrics#

Contribution module of metrics

class ignite.contrib.metrics.AveragePrecision(activation=None, output_transform=<function AveragePrecision.<lambda>>)[source]#

Computes Average Precision accumulating predictions and the ground-truth during an epoch and applying sklearn.metrics.average_precision_score .

Parameters

output_transform (callable, optional) – a callable that is used to transform the Engine’s process_function’s output into the form expected by the metric. This can be useful if, for example, you have a multi-output model and you want to compute the metric with respect to one of the outputs.

AveragePrecision expects y to be comprised of 0’s and 1’s. y_pred must either be probability estimates or confidence values. To apply an activation to y_pred, use output_transform as shown below:

def activated_output_transform(output):
y_pred, y = output
y_pred = torch.softmax(y_pred)
return y_pred, y

avg_precision = AveragePrecision(activated_output_transform)

class ignite.contrib.metrics.ROC_AUC(output_transform=<function ROC_AUC.<lambda>>)[source]#

Computes Area Under the Receiver Operating Characteristic Curve (ROC AUC) accumulating predictions and the ground-truth during an epoch and applying sklearn.metrics.roc_auc_score .

Parameters

output_transform (callable, optional) – a callable that is used to transform the Engine’s process_function’s output into the form expected by the metric. This can be useful if, for example, you have a multi-output model and you want to compute the metric with respect to one of the outputs.

ROC_AUC expects y to be comprised of 0’s and 1’s. y_pred must either be probability estimates or confidence values. To apply an activation to y_pred, use output_transform as shown below:

def activated_output_transform(output):
y_pred, y = output
y_pred = torch.sigmoid(y_pred)
return y_pred, y

roc_auc = ROC_AUC(activated_output_transform)


## Regression metrics#

Module ignite.contrib.metrics.regression provides implementations of metrics useful for regression tasks. Definitions of metrics are based on Botchkarev 2018, page 30 “Appendix 2. Metrics mathematical definitions”.

Complete list of metrics:

class ignite.contrib.metrics.regression.CanberraMetric(output_transform=<function Metric.<lambda>>)[source]#

Calculates the Canberra Metric.

$\text{CM} = \sum_{j=1}^n\frac{|A_j - P_j|}{A_j + P_j}$

where, $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape (N, ) or (N, 1).

class ignite.contrib.metrics.regression.FractionalAbsoluteError(output_transform=<function Metric.<lambda>>)[source]#

Calculates the Fractional Absolute Error.

$\text{FAE} = \frac{1}{n}\sum_{j=1}^n\frac{2 |A_j - P_j|}{|A_j| + |P_j|}$

where, $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape (N, ) or (N, 1).

class ignite.contrib.metrics.regression.FractionalBias(output_transform=<function Metric.<lambda>>)[source]#

Calculates the Fractional Bias:

$\text{FB} = \frac{1}{n}\sum_{j=1}^n\frac{2 (A_j - P_j)}{A_j + P_j}$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape (N, ) or (N, 1).

class ignite.contrib.metrics.regression.GeometricMeanAbsoluteError(output_transform=<function Metric.<lambda>>)[source]#

Calculates the Geometric Mean Absolute Error.

$\text{GMAE} = \exp(\frac{1}{n}\sum_{j=1}^n\ln(|A_j - P_j|)$)

where, $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape (N, ) or (N, 1).

class ignite.contrib.metrics.regression.GeometricMeanRelativeAbsoluteError(output_transform=<function Metric.<lambda>>)[source]#

Calculates the Geometric Mean Relative Absolute Error:

$\text{GMRAE} = \exp(\frac{1}{n}\sum_{j=1}^n \ln\frac{|A_j - P_j|}{|A_j - \bar{A}|})$

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape (N, ) or (N, 1).

class ignite.contrib.metrics.regression.ManhattanDistance(output_transform=<function Metric.<lambda>>)[source]#

Calculates the Manhattan Distance:

$\text{MD} = \sum_{j=1}^n (A_j - P_j)$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape (N, ) or (N, 1).

class ignite.contrib.metrics.regression.MaximumAbsoluteError(output_transform=<function Metric.<lambda>>)[source]#

Calculates the Maximum Absolute Error:

$\text{MaxAE} = \max_{j=1,n} \left( \lvert A_j-P_j \rvert \right)$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape (N, ) or (N, 1).

class ignite.contrib.metrics.regression.MeanAbsoluteRelativeError(output_transform=<function Metric.<lambda>>)[source]#

Calculate Mean Absolute Relative Error:

$\text{MARE} = \frac{1}{n}\sum_{j=1}^n\frac{\left|A_j-P_j\right|}{\left|A_j\right|}$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in the reference Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape (N, ) or (N, 1).

class ignite.contrib.metrics.regression.MeanError(output_transform=<function Metric.<lambda>>)[source]#

Calculates the Mean Error:

$\text{ME} = \frac{1}{n}\sum_{j=1}^n (A_j - P_j)$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in the reference Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape (N, ) or (N, 1).

class ignite.contrib.metrics.regression.MeanNormalizedBias(output_transform=<function Metric.<lambda>>)[source]#

Calculates the Mean Normalized Bias:

$\text{MNB} = \frac{1}{n}\sum_{j=1}^n\frac{A_j - P_j}{A_j}$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in the reference Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape (N, ) or (N, 1).

class ignite.contrib.metrics.regression.MedianAbsoluteError(output_transform=<function MedianAbsoluteError.<lambda>>)[source]#

Calculates the Median Absolute Error:

$\text{MdAE} = \text{MD}_{j=1,n} \left( |A_j - P_j| \right)$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape (N, ) or (N, 1) and of type float32.

Warning

Current implementation stores all input data (output and target) in as tensors before computing a metric. This can potentially lead to a memory error if the input data is larger than available RAM.

class ignite.contrib.metrics.regression.MedianAbsolutePercentageError(output_transform=<function MedianAbsolutePercentageError.<lambda>>)[source]#

Calculates the Median Absolute Percentage Error:

$\text{MdAPE} = 100 \cdot \text{MD}_{j=1,n} \left( \frac{|A_j - P_j|}{|A_j|} \right)$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape (N, ) or (N, 1) and of type float32.

Warning

Current implementation stores all input data (output and target) in as tensors before computing a metric. This can potentially lead to a memory error if the input data is larger than available RAM.

class ignite.contrib.metrics.regression.MedianRelativeAbsoluteError(output_transform=<function MedianRelativeAbsoluteError.<lambda>>)[source]#

Calculates the Median Relative Absolute Error:

$\text{MdRAE} = \text{MD}_{j=1,n} \left( \frac{|A_j - P_j|}{|A_j - \bar{A}|} \right)$,

where $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape (N, ) or (N, 1) and of type float32.

Warning

Current implementation stores all input data (output and target) in as tensors before computing a metric. This can potentially lead to a memory error if the input data is larger than available RAM.

class ignite.contrib.metrics.regression.WaveHedgesDistance(output_transform=<function Metric.<lambda>>)[source]#

Calculates the Wave Hedges Distance.

$\text{WHD} = \sum_{j=1}^n\frac{|A_j - P_j|}{max(A_j, P_j)}$, where, $A_j$ is the ground truth and $P_j$ is the predicted value.

More details can be found in Botchkarev 2018.

• update must receive output of the form (y_pred, y).

• y and y_pred must be of same shape (N, ) or (N, 1).