Fbeta#

ignite.metrics.Fbeta(beta, average=True, precision=None, recall=None, output_transform=None, device=device(type='cpu'))[source]#

Calculates F-beta score.

F_\beta = \left( 1 + \beta^2 \right) * \frac{ \text{precision} * \text{recall} } { \left( \beta^2 * \text{precision} \right) + \text{recall} }

where $\beta$ is a positive real factor.

update must receive output of the form (y_pred, y) or {'y_pred': y_pred, 'y': y}.
y_pred must be in the following shape (batch_size, num_categories, …) or (batch_size, …).
y must be in the following shape (batch_size, …).

Parameters

beta (float) – weight of precision in harmonic mean
average (bool) – if True, F-beta score is computed as the unweighted average (across all classes in multiclass case), otherwise, returns a tensor with F-beta score for each class in multiclass case.
precision (Optional[Precision]) – precision object metric with average=False to compute F-beta score
recall (Optional[Recall]) – recall object metric with average=False to compute F-beta score
output_transform (Optional[Callable]) – a callable that is used to transform the Engine’s process_function’s output into the form expected by the metric. It is used only if precision or recall are not provided.
device (Union[str, device]) – specifies which device updates are accumulated on. Setting the metric’s device to be the same as your update arguments ensures the update method is non-blocking. By default, CPU.

Returns

MetricsLambda, F-beta metric

Return type

MetricsLambda

Examples

Binary case

P = Precision(average=False)
R = Recall(average=False)
metric = Fbeta(beta=1.0, precision=P, recall=R)
metric.attach(default_evaluator, "f-beta")
y_true = torch.Tensor([1, 0, 1, 1, 0, 1])
y_pred = torch.Tensor([1, 0, 1, 0, 1, 1])
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics["f-beta"])

0.7499...

Multiclass case

P = Precision(average=False)
R = Recall(average=False)
metric = Fbeta(beta=1.0, precision=P, recall=R)
metric.attach(default_evaluator, "f-beta")
y_true = torch.Tensor([2, 0, 2, 1, 0, 1]).long()
y_pred = torch.Tensor([
    [0.0266, 0.1719, 0.3055],
    [0.6886, 0.3978, 0.8176],
    [0.9230, 0.0197, 0.8395],
    [0.1785, 0.2670, 0.6084],
    [0.8448, 0.7177, 0.7288],
    [0.7748, 0.9542, 0.8573],
])
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics["f-beta"])

0.5222...

F-beta can be computed for each class as done below:

P = Precision(average=False)
R = Recall(average=False)
metric = Fbeta(beta=1.0, average=False, precision=P, recall=R)
metric.attach(default_evaluator, "f-beta")
y_true = torch.Tensor([2, 0, 2, 1, 0, 1]).long()
y_pred = torch.Tensor([
    [0.0266, 0.1719, 0.3055],
    [0.6886, 0.3978, 0.8176],
    [0.9230, 0.0197, 0.8395],
    [0.1785, 0.2670, 0.6084],
    [0.8448, 0.7177, 0.7288],
    [0.7748, 0.9542, 0.8573],
])
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics["f-beta"])

tensor([0.5000, 0.6667, 0.4000], dtype=torch.float64)

The elements of y and y_pred should have 0 or 1 values. Thresholding of predictions can be done as below:

def thresholded_output_transform(output):
    y_pred, y = output
    y_pred = torch.round(y_pred)
    return y_pred, y

P = Precision(average=False, output_transform=thresholded_output_transform)
R = Recall(average=False, output_transform=thresholded_output_transform)
metric = Fbeta(beta=1.0, precision=P, recall=R)
metric.attach(default_evaluator, "f-beta")
y_true = torch.Tensor([1, 0, 1, 1, 0, 1])
y_pred = torch.Tensor([0.6, 0.2, 0.9, 0.4, 0.7, 0.65])
state = default_evaluator.run([[y_pred, y_true]])
print(state.metrics["f-beta"])

0.7499...