torcheval.metrics.BinaryAUPRC

class torcheval.metrics.BinaryAUPRC(*, num_tasks: int = 1, device: device | None = None)

Compute AUPRC, also called Average Precision, which is the area under the Precision-Recall Curve, for binary classification.

Precision is defined as \(\frac{T_p}{T_p+F_p}\), it is the probability that a positive prediction from the model is a true positive. Recall is defined as \(\frac{T_p}{T_p+F_n}\), it is the probability that a true positive is predicted to be positive by the model.

The precision-recall curve plots the recall on the x axis and the precision on the y axis, both of which are bounded between 0 and 1. This function returns the area under that graph. If the area is near one, the model supports a threshold which correctly identifies a high percentage of true positives while also rejecting enough false examples so that most of the true predictions are true positives.

Binary auprc supports multiple tasks, if the input and target tensors are 2 dimensional each row will be evaluated as if it were an independent instance of binary auprc.

The functional version of this metric is torcheval.metrics.functional.binary_auprc().

Parameters:

num_tasks (int) – Number of tasks that need BinaryAUPRC calculation. Default value is 1. Binary AUPRC for each task will be calculated independently. Results are equivalent to running Binary AUPRC calculation for each row.

Examples:

>>> import torch
>>> from torcheval import BinaryAUPRC
>>> metric = BinaryAUPRC()
>>> input = torch.tensor([0.1, 0.5, 0.7, 0.8])
>>> target = torch.tensor([1, 0, 1, 1])
>>> metric.update(input, target)
>>> metric.compute()
tensor(0.9167) # scalar returned with 1D input tensors

# with logits
>>> metric = BinaryAUPRC()
>>> input = torch.tensor([[.5, 2]])
>>> target = torch.tensor([[0, 0]])
>>> metric.update(input, target)
>>> metric.compute()
tensor([-0.])
>>> input = torch.tensor([[2, 1.5]])
>>> target = torch.tensor([[1, 0]])
>>> metric.update(input, target)
>>> metric.compute()
tensor([0.5000]) # 1D tensor returned with 2D input tensors

# multiple tasks
>>> metric = BinaryAUPRC(num_tasks=3)
>>> input = torch.tensor([[0.1, 0, 0.1, 0], [0, 1, 0.2, 0], [0, 0, 0.7, 1]])
>>> target = torch.tensor([[1, 0, 0, 0], [0, 1, 0, 0], [0, 0, 1, 1]])
>>> metric.update(input, target)
>>> metric.compute()
tensor([0.5000, 1.0000, 1.0000])

__init__(*, num_tasks: int = 1, device: device | None = None) → None

Initialize a metric object and its internal states.

Use self._add_state() to initialize state variables of your metric class. The state variables should be either torch.Tensor, a list of torch.Tensor, a dictionary with torch.Tensor as values, or a deque of torch.Tensor.

Methods

__init__(*[, num_tasks, device])	Initialize a metric object and its internal states.
compute()	Implement this method to compute and return the final metric value from state variables.
load_state_dict(state_dict[, strict])	Loads metric state variables from state_dict.
merge_state(metrics)	Implement this method to update the current metric's state variables to be the merged states of the current metric and input metrics.
reset()	Reset the metric state variables to their default value.
state_dict()	Save metric state variables in state_dict.
to(device, *args, **kwargs)	Move tensors in metric state variables to device.
update(input, target)	Update states with the ground truth labels and predictions.

Attributes

device

The last input device of Metric.to().