SoftUpdate

class torchrl.objectives.SoftUpdate(loss_module: Union['DQNLoss', 'DDPGLoss', 'SACLoss', 'REDQLoss', 'TD3Loss'], *, eps: float = None, tau: Optional[float] = None)

A soft-update class for target network update in Double DQN/DDPG.

This was proposed in “CONTINUOUS CONTROL WITH DEEP REINFORCEMENT LEARNING”, https://arxiv.org/pdf/1509.02971.pdf

One and only one decay factor (tau or eps) must be specified.

Parameters:

• loss_module (DQNLoss or DDPGLoss) – loss module where the target network should be updated.

• eps (scalar) –

  epsilon in the update equation: .. math:

  \theta_t = \theta_{t-1} * \epsilon + \theta_t * (1-\epsilon)
  

  Exclusive with tau.

• tau (scalar) – Polyak tau. It is equal to 1-eps, and exclusive with it.