ObjectDetectionAvgPrecisionRecall

class ignite.metrics.ObjectDetectionAvgPrecisionRecall(iou_thresholds=None, rec_thresholds=None, num_classes=80, max_detections_per_image_per_class=100, area_range='all', output_transform=<function ObjectDetectionAvgPrecisionRecall.<lambda>>, device=device(type='cpu'), skip_unrolling=False)

Calculate mean average precision & recall for evaluating an object detector in the COCO way.

Average precision is computed by averaging precision over increasing levels of recall thresholds. In COCO, the maximum precision across thresholds greater or equal a recall threshold is considered as the average summation operand; In other words, the precision peek across lower or equal sensivity levels is used for a recall threshold:

$$\text{Average Precision} = \sum_{k=1}^{\#rec\_thresholds} (r_k - r_{k-1}) max(P_{k:})$$

Average recall is the detector’s maximum recall, considering all matched detections as TP, averaged over classes.

Parameters

• iou_thresholds (Optional[Union[Sequence[float], Tensor]]) – sequence of IoU thresholds to be considered for computing mean average precision & recall. Values should be between 0 and 1. If not given, COCO’s default (.5, .55, …, .95) would be used.

• rec_thresholds (Optional[Union[Sequence[float], Tensor]]) – sequence of recall thresholds to be considered for computing mean average precision. Values should be between 0 and 1. If not given, COCO’s default (.0, .01, .02, …, 1.) would be used.

• num_classes (int) – number of categories. Default is 80, that of the COCO dataset.

• area_range (Literal['small', 'medium', 'large', 'all']) – area range which only objects therein are considered in evaluation. By default, ‘all’.

• max_detections_per_image_per_class (int) – maximum number of detections per class in each image to consider for evaluation. The most confident ones are selected.

• output_transform (Callable) – a callable that is used to transform the Engine’s process_function’s output into the form expected by the metric. An already provided example is coco_tensor_list_to_dict_list() which accepts y_pred and y as lists of tensors and transforms them to the expected format. Default is the identity function.

• device (Union[str, device]) – specifies which device updates are accumulated on. Setting the metric’s device to be the same as your update arguments ensures the update method is non-blocking. By default, CPU.

• skip_unrolling (bool) – specifies whether output should be unrolled before being fed to update method. Should be true for multi-output model, for example, if y_pred and y contain multi-ouput as (y_pred_a, y_pred_b) and (y_a, y_b), in which case the update method is called for (y_pred_a, y_a) and (y_pred_b, y_b).Alternatively, output_transform can be used to handle this.

New in version 0.5.2.

Methods

compute	Computes the metric based on its accumulated state.
reset	Resets the metric to its initial state.
update	Metric update method using prediction and target.

compute()

Computes the metric based on its accumulated state.

By default, this is called at the end of each epoch.

Returns

the actual quantity of interest. However, if a Mapping is returned, it will be (shallow) flattened into engine.state.metrics when completed() is called.

Return type

Any

Raises

NotComputableError – raised when the metric cannot be computed.

reset()

Resets the metric to its initial state.

By default, this is called at the start of each epoch.

Return type

None

update(output)

Metric update method using prediction and target.

Parameters

output (Tuple[List[Dict[str, Tensor]], List[Dict[str, Tensor]]]) –

a tuple, (y_pred, y), of two same-length lists, each one containing str-to-tensor dictionaries whose items is as follows. N_det and N_gt are number of detections and ground truths for a sample respectively.

y_pred items
Key	Value shape	Description
’bbox’	(Ndet, 4)	Bounding boxes of form (x1, y1, x2, y2) containing top left and bottom right coordinates.
’scores’	(Ndet,)	Confidence score of detections.
’labels’	(Ndet,)	Predicted category number of detections in torch.long dtype.

y items
Key	Value shape	Description
’bbox’	(Ngt, 4)	Bounding boxes of form (x1, y1, x2, y2) containing top left and bottom right coordinates.
’labels’	(Ngt,)	Category number of ground truths in torch.long dtype.
’iscrowd’	(Ngt,)	Whether ground truth boxes are crowd ones or not. This key is optional.

Return type

None