rlu_atari_checkpoints_ordered

• Description:

RL Unplugged is suite of benchmarks for offline reinforcement learning. The RL Unplugged is designed around the following considerations: to facilitate ease of use, we provide the datasets with a unified API which makes it easy for the practitioner to work with all data in the suite once a general pipeline has been established.

The datasets follow the RLDS format to represent steps and episodes.

We are releasing a large and diverse dataset of gameplay following the protocol described by Agarwal et al., 2020, which can be used to evaluate several discrete offline RL algorithms. The dataset is generated by running an online DQN agent and recording transitions from its replay during training with sticky actions Machado et al., 2018. As stated in Agarwal et al., 2020, for each game we use data from five runs with 50 million transitions each. We release datasets for 46 Atari games. For details on how the dataset was generated, please refer to the paper. Please see this note about the ROM versions used to generate the datasets.

Atari is a standard RL benchmark. We recommend you to try offline RL methods on Atari if you are interested in comparing your approach to other state of the art offline RL methods with discrete actions.

The reward of each step is clipped (obtained with [-1, 1] clipping) and the episode includes the sum of the clipped reward per episode.

Each of the configurations is broken into splits. Splits correspond to checkpoints of 1M steps (note that the number of episodes may difer). Checkpoints are ordered in time (so checkpoint 0 ran before checkpoint 1).

Episodes within each split are ordered. Check https://www.tensorflow.org/datasets/determinism if you want to ensure that you read episodes in order.

This dataset corresponds to the one used in the DQN replay paper. https://research.google/tools/datasets/dqn-replay/

• Homepage: https://github.com/deepmind/deepmind-research/tree/master/rl_unplugged

• Source code: tfds.rl_unplugged.rlu_atari_checkpoints_ordered.RluAtariCheckpointsOrdered

• Versions:

  • 1.0.0: Initial release.
  • 1.1.0 (default): Removed redundant clipped reward fields.

• Download size: Unknown size

• Auto-cached (documentation): No

• Feature structure:

FeaturesDict({
    'checkpoint_id': int64,
    'episode_id': int64,
    'episode_return': Scalar(shape=(), dtype=float32, description=Sum of the clipped rewards.),
    'steps': Dataset({
        'action': int64,
        'discount': float32,
        'is_first': bool,
        'is_last': bool,
        'is_terminal': bool,
        'observation': Image(shape=(84, 84, 1), dtype=uint8),
        'reward': Scalar(shape=(), dtype=float32, description=Clipped reward.),
    }),
})

• Feature documentation:

• Supervised keys (See as_supervised doc): None

• Figure (tfds.show_examples): Not supported.

• Citation:

@misc{gulcehre2020rl,
    title={RL Unplugged: Benchmarks for Offline Reinforcement Learning},
    author={Caglar Gulcehre and Ziyu Wang and Alexander Novikov and Tom Le Paine
        and  Sergio Gómez Colmenarejo and Konrad Zolna and Rishabh Agarwal and
        Josh Merel and Daniel Mankowitz and Cosmin Paduraru and Gabriel
        Dulac-Arnold and Jerry Li and Mohammad Norouzi and Matt Hoffman and
        Ofir Nachum and George Tucker and Nicolas Heess and Nando deFreitas},
    year={2020},
    eprint={2006.13888},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

rlu_atari_checkpoints_ordered/Alien_run_1 (default config)

• Dataset size: 73.90 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Alien_run_2

• Dataset size: 74.13 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Alien_run_3

• Dataset size: 74.02 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Alien_run_4

• Dataset size: 74.43 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Alien_run_5

• Dataset size: 74.09 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Amidar_run_1

• Dataset size: 76.91 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Amidar_run_2

• Dataset size: 76.70 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Amidar_run_3

• Dataset size: 77.03 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Amidar_run_4

• Dataset size: 77.16 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Amidar_run_5

• Dataset size: 77.42 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Assault_run_1

• Dataset size: 28.70 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Assault_run_2

• Dataset size: 28.62 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Assault_run_3

• Dataset size: 28.66 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Assault_run_4

• Dataset size: 28.64 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Assault_run_5

• Dataset size: 28.63 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Asterix_run_1

• Dataset size: 46.55 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Asterix_run_2

• Dataset size: 46.64 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Asterix_run_3

• Dataset size: 46.45 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Asterix_run_4

• Dataset size: 46.57 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Asterix_run_5

• Dataset size: 46.60 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Atlantis_run_1

• Dataset size: 41.46 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Atlantis_run_2

• Dataset size: 41.25 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Atlantis_run_3

• Dataset size: 41.48 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Atlantis_run_4

• Dataset size: 41.54 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/Atlantis_run_5

• Dataset size: 41.48 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/BankHeist_run_1

• Dataset size: 71.93 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/BankHeist_run_2

• Dataset size: 71.93 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/BankHeist_run_3

• Dataset size: 71.82 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/BankHeist_run_4

• Dataset size: 71.93 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/BankHeist_run_5

• Dataset size: 71.96 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/BattleZone_run_1

• Dataset size: 57.82 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/BattleZone_run_2

• Dataset size: 57.79 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/BattleZone_run_3

• Dataset size: 57.80 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/BattleZone_run_4

• Dataset size: 57.73 GiB

• Splits:

• Examples (tfds.as_dataframe):

rlu_atari_checkpoints_ordered/BattleZone_run_5

• Dataset size: 57.83 GiB

• Splits: