ICM
ICM(
observation_space: gym.Space, action_space: gym.Space, device: str = 'cpu',
beta: float = 0.05, kappa: float = 2.5e-05, latent_dim: int = 128, lr: float = 0.001,
batch_size: int = 64
)
Curiosity-Driven Exploration by Self-Supervised Prediction. See paper: http://proceedings.mlr.press/v70/pathak17a/pathak17a.pdf
Args
- observation_space (Space) : The observation space of environment.
- action_space (Space) : The action space of environment.
- device (str) : Device (cpu, cuda, ...) on which the code should be run.
- beta (float) : The initial weighting coefficient of the intrinsic rewards.
- kappa (float) : The decay rate.
- latent_dim (int) : The dimension of encoding vectors.
- lr (float) : The learning rate.
- batch_size (int) : The batch size for update.
Returns
Instance of ICM.
Methods:
.compute_irs
Compute the intrinsic rewards for current samples.
Args
- samples (Dict) : The collected samples. A python dict like
{obs (n_steps, n_envs, obs_shape)
, actions (n_steps, n_envs, action_shape), rewards (n_steps, n_envs) , next_obs (n_steps, n_envs, *obs_shape) }. - step (int) : The global training step.
Returns
The intrinsic rewards.
.add
Add new samples to the intrinsic reward module.
.update
Update the intrinsic reward module if necessary.
Args
- samples : The collected samples. A python dict like
{obs (n_steps, n_envs, obs_shape)
, actions (n_steps, n_envs, action_shape), rewards (n_steps, n_envs) , next_obs (n_steps, n_envs, *obs_shape) }.
Returns
None