torcheval.metrics.Perplexity¶
-
class
torcheval.metrics.
Perplexity
(ignore_index: Optional[int] = None, device: Optional[device] = None)[source]¶ Perplexity measures how well a model predicts sample data. It is calculated by:
ppl = exp (sum of negative log likelihood / number of tokens)
Its functional version is
torcheval.metrics.functional.text.perplexity
.Parameters: ignore_index (Tensor) – if specified, the target class with ‘ignore_index’ will be ignored when calculating perplexity. The default value is None. Examples
>>> import torch >>> from torcheval.metrics.text import Perplexity
>>> metric=Perplexity() >>> input = torch.tensor([[[0.3659, 0.7025, 0.3104]], [[0.0097, 0.6577, 0.1947]],[[0.5659, 0.0025, 0.0104]], [[0.9097, 0.0577, 0.7947]]]) >>> target = torch.tensor([[2], [1], [2], [1]]) >>> metric.update(input, target) >>> metric.compute() tensor(3.5257, dtype=torch.float64)
>>> metric=Perplexity(ignore_index=1) >>> input = torch.tensor([[[0.3659, 0.7025, 0.3104]], [[0.0097, 0.6577, 0.1947]],[[0.5659, 0.0025, 0.0104]], [[0.9097, 0.0577, 0.7947]]]) >>> target = torch.tensor([[2], [1], [2], [1]]) >>> metric.update(input, target) >>> metric.compute() tensor(3.6347, dtype=torch.float64)
>>> metric1=Perplexity() >>> input = torch.tensor([[[0.5659, 0.0025, 0.0104]], [[0.9097, 0.0577, 0.7947]]]) >>> target = torch.tensor([[2], [1], ]) >>> metric1.update(input, target) >>> metric1.compute() tensor(4.5051, dtype=torch.float64)
>>> metric2=Perplexity() >>> input = torch.tensor([[[0.3659, 0.7025, 0.3104]], [[0.0097, 0.6577, 0.1947]]]) >>> target = torch.tensor([[2], [1]]) >>> metric2.update(input, target) >>> metric2.compute()) tensor(2.7593, dtype=torch.float64)
>>> metric1.merge_state([metric2]) >>> metric1.compute()) tensor(3.5257, dtype=torch.float64)
-
__init__
(ignore_index: Optional[int] = None, device: Optional[device] = None) None [source]¶ Initialize a metric object and its internal states.
Use
self._add_state()
to initialize state variables of your metric class. The state variables should be eithertorch.Tensor
, a list oftorch.Tensor
, or a dictionary withtorch.Tensor
as values
Methods
__init__
([ignore_index, device])Initialize a metric object and its internal states. compute
()Calculates perplexity based on sum_log_probs and num_total. load_state_dict
(state_dict[, strict])Loads metric state variables from state_dict. merge_state
(metrics)Merge the metric state with its counterparts from other metric instances. reset
()Reset the metric state variables to their default value. state_dict
()Save metric state variables in state_dict. to
(device, *args, **kwargs)Move tensors in metric state variables to device. update
(input, target)Update the metric state with new inputs. Attributes
device
The last input device of Metric.to()
.-