OnlineDTLoss¶
- class torchrl.objectives.OnlineDTLoss(*args, **kwargs)[source]¶
TorchRL implementation of the Online Decision Transformer loss.
Presented in “Online Decision Transformer” <https://arxiv.org/abs/2202.05607>
- Parameters:
actor_network (ProbabilisticActor) – stochastic actor
- Keyword Arguments:
alpha_init (
float, optional) – initial entropy multiplier. Default is 1.0.min_alpha (
float, optional) – min value of alpha. Default is None (no minimum value).max_alpha (
float, optional) – max value of alpha. Default is None (no maximum value).fixed_alpha (bool, optional) – if
True, alpha will be fixed to its initial value. Otherwise, alpha will be optimized to match the ‘target_entropy’ value. Default isFalse.target_entropy (float or str, optional) – Target entropy for the stochastic policy. Default is “auto”, where target entropy is computed as
-prod(n_actions).samples_mc_entropy (int) – number of samples to estimate the entropy
reduction (str, optional) – Specifies the reduction to apply to the output:
"none"|"mean"|"sum"."none": no reduction will be applied,"mean": the sum of the output will be divided by the number of elements in the output,"sum": the output will be summed. Default:"mean".
