Skip to content

RND

source 

RND(
   observation_space: gym.Space, action_space: gym.Space, device: str = 'cpu',
   beta: float = 0.05, kappa: float = 2.5e-05, latent_dim: int = 128, lr: float = 0.001,
   batch_size: int = 64
)

Exploration by Random Network Distillation (RND). See paper: https://arxiv.org/pdf/1810.12894.pdf

Args

  • observation_space (Space) : The observation space of environment.
  • action_space (Space) : The action space of environment.
  • device (str) : Device (cpu, cuda, ...) on which the code should be run.
  • beta (float) : The initial weighting coefficient of the intrinsic rewards.
  • kappa (float) : The decay rate.
  • latent_dim (int) : The dimension of encoding vectors.
  • lr (float) : The learning rate.
  • batch_size (int) : The batch size for update.

Returns

Instance of RND.

Methods:

.compute_irs

source 

.compute_irs(
   samples: Dict, step: int = 0
)

Compute the intrinsic rewards for current samples.

Args

  • samples (Dict) : The collected samples. A python dict like {obs (n_steps, n_envs, obs_shape) , actions (n_steps, n_envs, action_shape) , rewards (n_steps, n_envs) , next_obs (n_steps, n_envs, *obs_shape) }.
  • step (int) : The global training step.

Returns

The intrinsic rewards.

.add

source 

.add(
   samples: Dict
)

Add new samples to the intrinsic reward module.

.update

source 

.update(
   samples: Dict
)

Update the intrinsic reward module if necessary.

Args

  • samples : The collected samples. A python dict like {obs (n_steps, n_envs, obs_shape) , actions (n_steps, n_envs, action_shape) , rewards (n_steps, n_envs) , next_obs (n_steps, n_envs, *obs_shape) }.

Returns

None