API Reference

• torchrl.collectors package
  • Collectors and batch size
  • Collectors and policy copies
  • Collectors and replay buffers interoperability
  • Single node data collectors
    • DataCollectorBase
    • SyncDataCollector
    • MultiSyncDataCollector
    • MultiaSyncDataCollector
    • aSyncDataCollector
  • Distributed data collectors
    • DistributedDataCollector
    • RPCDataCollector
    • DistributedSyncDataCollector
    • submitit_delayed_launcher
    • RayCollector
  • Helper functions
    • split_trajectories
• torchrl.data package
  • Replay Buffers
    • ReplayBuffer
    • PrioritizedReplayBuffer
    • TensorDictReplayBuffer
    • TensorDictPrioritizedReplayBuffer
  • Composable Replay Buffers
    • Supported data types and choosing a storage
    • Sampling and indexing
    • Sharing replay buffers across processes
    • Storing trajectories
    • Checkpointing Replay Buffers
  • TorchRL Episode Data Format (TED)
    • The Rationale Behind TED
    • The structure of TED
    • Flattening TED to reduce memory consumption
    • Special cases and footnotes
  • Datasets
    • Transforming datasets
    • Composing datasets
  • TensorSpec
    • TensorSpec
    • Binary
    • Bounded
    • Categorical
    • Composite
    • MultiCategorical
    • MultiOneHot
    • NonTensor
    • Stacked
    • StackedComposite
    • Unbounded
    • UnboundedContinuous
    • UnboundedDiscrete
    • BinaryDiscreteTensorSpec
    • BoundedTensorSpec
    • CompositeSpec
    • DiscreteTensorSpec
    • LazyStackedCompositeSpec
    • LazyStackedTensorSpec
    • MultiDiscreteTensorSpec
    • MultiOneHotDiscreteTensorSpec
    • NonTensorSpec
    • OneHotDiscreteTensorSpec
    • UnboundedContinuousTensorSpec
    • UnboundedDiscreteTensorSpec
  • Trees and Forests
    • BinaryToDecimal
    • HashToInt
    • QueryModule
    • RandomProjectionHash
    • SipHash
    • TensorDictMap
    • TensorMap
  • Reinforcement Learning From Human Feedback (RLHF)
    • PairwiseDataset
    • PromptData
    • PromptTensorDictTokenizer
    • RewardData
    • RolloutFromModel
    • TensorDictTokenizer
    • TokenizedDatasetLoader
    • create_infinite_iterator
    • get_dataloader
    • ConstantKLController
    • AdaptiveKLController
  • Utils
    • MultiStep
    • consolidate_spec
    • check_no_exclusive_keys
    • contains_lazy_spec
    • Nested2TED
    • Flat2TED
    • H5Combine
    • H5Split
    • TED2Flat
    • TED2Nested
    • MultiStepTransform
• torchrl.envs package
  • EnvBase
  • GymLikeEnv
  • EnvMetaData
  • Vectorized envs
    • SerialEnv
    • ParallelEnv
    • EnvCreator
  • Custom native TorchRL environments
    • PendulumEnv
    • TicTacToeEnv
  • Multi-agent environments
    • MarlGroupMapType
    • check_marl_grouping
  • Auto-resetting Envs
  • Dynamic Specs
  • Transforms
    • Forward and inverse transforms
    • Cloning transforms
  • Environments with masked actions
  • Recorders
    • Recording videos
  • Helpers
    • RandomPolicy
    • check_env_specs
    • exploration_type
    • get_available_libraries
    • make_composite_from_td
    • set_exploration_type
    • step_mdp
    • terminated_or_truncated
  • Domain-specific
    • ModelBasedEnvBase
    • DreamerEnv
    • DreamerDecoder
  • Libraries
    • BraxEnv
    • BraxWrapper
    • DMControlEnv
    • DMControlWrapper
    • GymEnv
    • GymWrapper
    • HabitatEnv
    • IsaacGymEnv
    • IsaacGymWrapper
    • JumanjiEnv
    • JumanjiWrapper
    • MeltingpotEnv
    • MeltingpotWrapper
    • MOGymEnv
    • MOGymWrapper
    • MultiThreadedEnv
    • MultiThreadedEnvWrapper
    • OpenMLEnv
    • OpenSpielWrapper
    • OpenSpielEnv
    • PettingZooEnv
    • PettingZooWrapper
    • RoboHiveEnv
    • SMACv2Env
    • SMACv2Wrapper
    • UnityMLAgentsEnv
    • UnityMLAgentsWrapper
    • VmasEnv
    • VmasWrapper
    • gym_backend
    • set_gym_backend
• torchrl.modules package
  • TensorDict modules: Actors, exploration, value models and generative models
    • TensorDictModules and SafeModules
    • Exploration wrappers and modules
    • Probabilistic actors
    • Q-Value actors
    • Value operators and joined models
    • Domain-specific TensorDict modules
  • Hooks
    • QValueHook
    • DistributionalQValueHook
  • Models
    • Regular modules
    • Algorithm-specific modules
    • Multi-agent-specific modules
  • Exploration
    • NoisyLinear
    • NoisyLazyLinear
    • reset_noise
  • Planners
    • CEMPlanner
    • MPCPlannerBase
    • MPPIPlanner
  • Distributions
    • Delta
    • IndependentNormal
    • NormalParamWrapper
    • TanhNormal
    • TruncatedNormal
    • TanhDelta
    • OneHotCategorical
    • MaskedCategorical
    • MaskedOneHotCategorical
  • Utils
    • mappings
    • inv_softplus
    • biased_softplus
    • get_primers_from_module
    • VmapModule
• torchrl.objectives package
  • torch.vmap and randomness
  • Training value functions
    • LossModule
  • DQN
    • DQNLoss
    • DistributionalDQNLoss
  • DDPG
    • DDPGLoss
  • SAC
    • SACLoss
    • DiscreteSACLoss
  • REDQ
    • REDQLoss
  • CrossQ
    • CrossQLoss
  • IQL
    • IQLLoss
    • DiscreteIQLLoss
  • CQL
    • CQLLoss
    • DiscreteCQLLoss
  • GAIL
    • GAILLoss
  • DT
    • DTLoss
    • OnlineDTLoss
  • TD3
    • TD3Loss
  • TD3+BC
    • TD3BCLoss
  • PPO
    • PPOLoss
    • ClipPPOLoss
    • KLPENPPOLoss
  • A2C
    • A2CLoss
  • Reinforce
    • ReinforceLoss
  • Dreamer
    • DreamerActorLoss
    • DreamerModelLoss
    • DreamerValueLoss
  • Multi-agent objectives
    • QMixer
  • Returns
    • ValueEstimatorBase
    • TD0Estimator
    • TD1Estimator
    • TDLambdaEstimator
    • GAE
    • td0_return_estimate
    • td0_advantage_estimate
    • td1_return_estimate
    • vec_td1_return_estimate
    • td1_advantage_estimate
    • vec_td1_advantage_estimate
    • td_lambda_return_estimate
    • vec_td_lambda_return_estimate
    • td_lambda_advantage_estimate
    • vec_td_lambda_advantage_estimate
    • generalized_advantage_estimate
    • vec_generalized_advantage_estimate
    • reward2go
  • Utils
    • distance_loss
    • hold_out_net
    • hold_out_params
    • next_state_value
    • SoftUpdate
    • HardUpdate
    • ValueEstimators
    • default_value_kwargs
• torchrl.trainers package
  • Checkpointing
  • Trainer and hooks
    • BatchSubSampler
    • ClearCudaCache
    • CountFramesLog
    • LogReward
    • OptimizerHook
    • Recorder
    • ReplayBufferTrainer
    • RewardNormalizer
    • SelectKeys
    • Trainer
    • TrainerHookBase
    • UpdateWeights
  • Builders
    • make_collector_offpolicy
    • make_collector_onpolicy
    • make_dqn_loss
    • make_replay_buffer
    • make_target_updater
    • make_trainer
    • parallel_env_constructor
    • sync_async_collector
    • sync_sync_collector
    • transformed_env_constructor
  • Utils
    • correct_for_frame_skip
    • get_stats_random_rollout
  • Loggers
    • Logger
    • CSVLogger
    • MLFlowLogger
    • TensorboardLogger
    • WandbLogger
    • get_logger
    • generate_exp_name
  • Recording utils
    • VideoRecorder
    • TensorDictRecorder
    • PixelRenderTransform
• torchrl._utils package
  • implement_for