API reference for cpprb

cpprb Package

cpprb: Fast Flexible Replay Buffer Library

cpprb provides replay buffer classes for reinforcement learning. Details are described at Home Page.

Examples

Replay Buffer classes can be imported from cpprb package.

>>> from cpprb import ReplayBuffer

These buffer classes can be created by specifying env_dict.

>>> buffer_size = 1e6
>>> env_dict = {"obs": {}, "act": {}, "rew": {}, "next_obs": {}, "done": {}}
>>> rb = ReplayBuffer(buffer_size, env_dict)

When adding transitions, all values must be passed as keyword arguments.

>>> rb.add(obs=1, act=1, rew=0.5, next_obs=2, done=0)

You can also add multiple transitions simultaneously.

>>> rb.add(obs=[1, 2], act=[1, 2], rew=[0.5, 0.3], next_obs=[2, 3], done=[0, 1])

At the episode end, users must call on_episode_end() method.

>>> rb.on_episode_end()

Transitions can be sampled according to these buffer’s algorithms (e.g. random).

>>> sample = rb.sample(32)

`create_buffer`(size[, env_dict, prioritized])	Create specified version of replay buffer
`train`(ReplayBuffer buffer, env, get_action, ...)	Train RL policy (model)

`ReplayBuffer`(size[, env_dict, next_of, ...])	Replay Buffer class to store transitions and to sample them randomly.
`PrioritizedReplayBuffer`(size[, env_dict, ...])	Prioritized Replay Buffer class to store transitions with priorities.
`MPReplayBuffer`(size[, env_dict, ...])	Multi-process support Replay Buffer class to store transitions and to sample them randomly.
`MPPrioritizedReplayBuffer`(size[, env_dict, ...])	Multi-process support Prioritized Replay Buffer class to store transitions with priorities.
`ReverseReplayBuffer`(size[, env_dict, stride])	Replay Buffer class for Reverse Experience Replay (RER)
`LaBERmean`(batch_size[, m, eps])	Helper class for Large Batch Experience Replay (LaBER)
`LaBERlazy`(batch_size[, m, eps])	Helper class for Large Batch Experience Replay (LaBER)
`LaBERmax`(batch_size[, m, eps])	Helper class for Large Batch Experience Replay (LaBER)
`HindsightReplayBuffer`(size, env_dict, ...[, ...])	Replay Buffer class for Hindsight Experience Replay (HER)