Shortcuts

torcheval.metrics.Perplexity

class torcheval.metrics.Perplexity(ignore_index: Optional[int] = None, device: Optional[device] = None)[source]

Perplexity measures how well a model predicts sample data. It is calculated by:

ppl = exp (sum of negative log likelihood / number of tokens)

Its functional version is torcheval.metrics.functional.text.perplexity.

Parameters:ignore_index (Tensor) – if specified, the target class with ‘ignore_index’ will be ignored when calculating perplexity. The default value is None.

Examples

>>> import torch
>>> from torcheval.metrics.text import Perplexity
>>> metric=Perplexity()
>>> input = torch.tensor([[[0.3659, 0.7025, 0.3104]], [[0.0097, 0.6577, 0.1947]],[[0.5659, 0.0025, 0.0104]], [[0.9097, 0.0577, 0.7947]]])
>>> target = torch.tensor([[2],  [1], [2],  [1]])
>>> metric.update(input, target)
>>> metric.compute()
tensor(3.5257, dtype=torch.float64)
>>> metric=Perplexity(ignore_index=1)
>>> input = torch.tensor([[[0.3659, 0.7025, 0.3104]], [[0.0097, 0.6577, 0.1947]],[[0.5659, 0.0025, 0.0104]], [[0.9097, 0.0577, 0.7947]]])
>>> target = torch.tensor([[2],  [1], [2],  [1]])
>>> metric.update(input, target)
>>> metric.compute()
tensor(3.6347, dtype=torch.float64)
>>> metric1=Perplexity()
>>> input = torch.tensor([[[0.5659, 0.0025, 0.0104]], [[0.9097, 0.0577, 0.7947]]])
>>> target = torch.tensor([[2],  [1], ])
>>> metric1.update(input, target)
>>> metric1.compute()
tensor(4.5051, dtype=torch.float64)
>>> metric2=Perplexity()
>>> input = torch.tensor([[[0.3659, 0.7025, 0.3104]], [[0.0097, 0.6577, 0.1947]]])
>>> target = torch.tensor([[2],  [1]])
>>> metric2.update(input, target)
>>> metric2.compute())
tensor(2.7593, dtype=torch.float64)
>>> metric1.merge_state([metric2])
>>> metric1.compute())
tensor(3.5257, dtype=torch.float64)
__init__(ignore_index: Optional[int] = None, device: Optional[device] = None) None[source]

Initialize a metric object and its internal states.

Use self._add_state() to initialize state variables of your metric class. The state variables should be either torch.Tensor, a list of torch.Tensor, or a dictionary with torch.Tensor as values

Methods

__init__([ignore_index, device]) Initialize a metric object and its internal states.
compute() Calculates perplexity based on sum_log_probs and num_total.
load_state_dict(state_dict[, strict]) Loads metric state variables from state_dict.
merge_state(metrics) Merge the metric state with its counterparts from other metric instances.
reset() Reset the metric state variables to their default value.
state_dict() Save metric state variables in state_dict.
to(device, *args, **kwargs) Move tensors in metric state variables to device.
update(input, target) Update the metric state with new inputs.

Attributes

device The last input device of Metric.to().

Docs

Access comprehensive developer documentation for PyTorch

View Docs

Tutorials

Get in-depth tutorials for beginners and advanced developers

View Tutorials

Resources

Find development resources and get your questions answered

View Resources