Google I/O is a wrap! Catch up on TensorFlow sessions View sessions

robomimic_ph

  • Description:

The Proficient Human datasets were collected by 1 proficient operator using the RoboTurk platform (with the exception of Transport, which had 2 proficient operators working together). Each dataset consists of 200 successful trajectories.

Each task has two versions: one with low dimensional observations (low_dim), and one with images (image).

The datasets follow the RLDS format to represent steps and episodes.

Split Examples
'train' 200
@inproceedings{robomimic2021,
  title={What Matters in Learning from Offline Human Demonstrations for Robot Manipulation},
  author={Ajay Mandlekar and Danfei Xu and Josiah Wong and Soroush Nasiriany
          and Chen Wang and Rohun Kulkarni and Li Fei-Fei and Silvio Savarese
          and Yuke Zhu and Roberto Mart'{i}n-Mart'{i}n},
  booktitle={Conference on Robot Learning},
  year={2021}
}

robomimic_ph/lift_low_dim (default config)

  • Download size: 17.69 MiB

  • Dataset size: 8.50 MiB

  • Auto-cached (documentation): Yes

  • Feature structure:

FeaturesDict({
    '20_percent': tf.bool,
    '20_percent_train': tf.bool,
    '20_percent_valid': tf.bool,
    '50_percent': tf.bool,
    '50_percent_train': tf.bool,
    '50_percent_valid': tf.bool,
    'episode_id': tf.string,
    'horizon': tf.int32,
    'steps': Dataset({
        'action': Tensor(shape=(7,), dtype=tf.float64),
        'discount': tf.int32,
        'is_first': tf.bool,
        'is_last': tf.bool,
        'is_terminal': tf.bool,
        'observation': FeaturesDict({
            'object': Tensor(shape=(10,), dtype=tf.float64),
            'robot0_eef_pos': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eef_quat': Tensor(shape=(4,), dtype=tf.float64),
            'robot0_eef_vel_ang': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eef_vel_lin': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_gripper_qpos': Tensor(shape=(2,), dtype=tf.float64),
            'robot0_gripper_qvel': Tensor(shape=(2,), dtype=tf.float64),
            'robot0_joint_pos': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_pos_cos': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_pos_sin': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_vel': Tensor(shape=(7,), dtype=tf.float64),
        }),
        'reward': tf.float64,
        'states': Tensor(shape=(32,), dtype=tf.float64),
    }),
    'train': tf.bool,
    'valid': tf.bool,
})
  • Feature documentation:
Feature Class Shape Dtype Description
FeaturesDict
20_percent Tensor tf.bool
20_percent_train Tensor tf.bool
20_percent_valid Tensor tf.bool
50_percent Tensor tf.bool
50_percent_train Tensor tf.bool
50_percent_valid Tensor tf.bool
episode_id Tensor tf.string
horizon Tensor tf.int32
steps Dataset
steps/action Tensor (7,) tf.float64
steps/discount Tensor tf.int32
steps/is_first Tensor tf.bool
steps/is_last Tensor tf.bool
steps/is_terminal Tensor tf.bool
steps/observation FeaturesDict
steps/observation/object Tensor (10,) tf.float64
steps/observation/robot0_eef_pos Tensor (3,) tf.float64
steps/observation/robot0_eef_quat Tensor (4,) tf.float64
steps/observation/robot0_eef_vel_ang Tensor (3,) tf.float64
steps/observation/robot0_eef_vel_lin Tensor (3,) tf.float64
steps/observation/robot0_gripper_qpos Tensor (2,) tf.float64
steps/observation/robot0_gripper_qvel Tensor (2,) tf.float64
steps/observation/robot0_joint_pos Tensor (7,) tf.float64
steps/observation/robot0_joint_pos_cos Tensor (7,) tf.float64
steps/observation/robot0_joint_pos_sin Tensor (7,) tf.float64
steps/observation/robot0_joint_vel Tensor (7,) tf.float64
steps/reward Tensor tf.float64
steps/states Tensor (32,) tf.float64
train Tensor tf.bool
valid Tensor tf.bool

robomimic_ph/lift_image

  • Download size: 798.43 MiB

  • Dataset size: 114.47 MiB

  • Auto-cached (documentation): Yes

  • Feature structure:

FeaturesDict({
    '20_percent': tf.bool,
    '20_percent_train': tf.bool,
    '20_percent_valid': tf.bool,
    '50_percent': tf.bool,
    '50_percent_train': tf.bool,
    '50_percent_valid': tf.bool,
    'episode_id': tf.string,
    'horizon': tf.int32,
    'steps': Dataset({
        'action': Tensor(shape=(7,), dtype=tf.float64),
        'discount': tf.int32,
        'is_first': tf.bool,
        'is_last': tf.bool,
        'is_terminal': tf.bool,
        'observation': FeaturesDict({
            'agentview_image': Image(shape=(84, 84, 3), dtype=tf.uint8),
            'object': Tensor(shape=(10,), dtype=tf.float64),
            'robot0_eef_pos': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eef_quat': Tensor(shape=(4,), dtype=tf.float64),
            'robot0_eef_vel_ang': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eef_vel_lin': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eye_in_hand_image': Image(shape=(84, 84, 3), dtype=tf.uint8),
            'robot0_gripper_qpos': Tensor(shape=(2,), dtype=tf.float64),
            'robot0_gripper_qvel': Tensor(shape=(2,), dtype=tf.float64),
            'robot0_joint_pos': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_pos_cos': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_pos_sin': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_vel': Tensor(shape=(7,), dtype=tf.float64),
        }),
        'reward': tf.float64,
        'states': Tensor(shape=(32,), dtype=tf.float64),
    }),
    'train': tf.bool,
    'valid': tf.bool,
})
  • Feature documentation:
Feature Class Shape Dtype Description
FeaturesDict
20_percent Tensor tf.bool
20_percent_train Tensor tf.bool
20_percent_valid Tensor tf.bool
50_percent Tensor tf.bool
50_percent_train Tensor tf.bool
50_percent_valid Tensor tf.bool
episode_id Tensor tf.string
horizon Tensor tf.int32
steps Dataset
steps/action Tensor (7,) tf.float64
steps/discount Tensor tf.int32
steps/is_first Tensor tf.bool
steps/is_last Tensor tf.bool
steps/is_terminal Tensor tf.bool
steps/observation FeaturesDict
steps/observation/agentview_image Image (84, 84, 3) tf.uint8
steps/observation/object Tensor (10,) tf.float64
steps/observation/robot0_eef_pos Tensor (3,) tf.float64
steps/observation/robot0_eef_quat Tensor (4,) tf.float64
steps/observation/robot0_eef_vel_ang Tensor (3,) tf.float64
steps/observation/robot0_eef_vel_lin Tensor (3,) tf.float64
steps/observation/robot0_eye_in_hand_image Image (84, 84, 3) tf.uint8
steps/observation/robot0_gripper_qpos Tensor (2,) tf.float64
steps/observation/robot0_gripper_qvel Tensor (2,) tf.float64
steps/observation/robot0_joint_pos Tensor (7,) tf.float64
steps/observation/robot0_joint_pos_cos Tensor (7,) tf.float64
steps/observation/robot0_joint_pos_sin Tensor (7,) tf.float64
steps/observation/robot0_joint_vel Tensor (7,) tf.float64
steps/reward Tensor tf.float64
steps/states Tensor (32,) tf.float64
train Tensor tf.bool
valid Tensor tf.bool

robomimic_ph/can_low_dim

  • Download size: 43.38 MiB

  • Dataset size: 27.73 MiB

  • Auto-cached (documentation): Yes

  • Feature structure:

FeaturesDict({
    '20_percent': tf.bool,
    '20_percent_train': tf.bool,
    '20_percent_valid': tf.bool,
    '50_percent': tf.bool,
    '50_percent_train': tf.bool,
    '50_percent_valid': tf.bool,
    'episode_id': tf.string,
    'horizon': tf.int32,
    'steps': Dataset({
        'action': Tensor(shape=(7,), dtype=tf.float64),
        'discount': tf.int32,
        'is_first': tf.bool,
        'is_last': tf.bool,
        'is_terminal': tf.bool,
        'observation': FeaturesDict({
            'object': Tensor(shape=(14,), dtype=tf.float64),
            'robot0_eef_pos': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eef_quat': Tensor(shape=(4,), dtype=tf.float64),
            'robot0_eef_vel_ang': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eef_vel_lin': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_gripper_qpos': Tensor(shape=(2,), dtype=tf.float64),
            'robot0_gripper_qvel': Tensor(shape=(2,), dtype=tf.float64),
            'robot0_joint_pos': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_pos_cos': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_pos_sin': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_vel': Tensor(shape=(7,), dtype=tf.float64),
        }),
        'reward': tf.float64,
        'states': Tensor(shape=(71,), dtype=tf.float64),
    }),
    'train': tf.bool,
    'valid': tf.bool,
})
  • Feature documentation:
Feature Class Shape Dtype Description
FeaturesDict
20_percent Tensor tf.bool
20_percent_train Tensor tf.bool
20_percent_valid Tensor tf.bool
50_percent Tensor tf.bool
50_percent_train Tensor tf.bool
50_percent_valid Tensor tf.bool
episode_id Tensor tf.string
horizon Tensor tf.int32
steps Dataset
steps/action Tensor (7,) tf.float64
steps/discount Tensor tf.int32
steps/is_first Tensor tf.bool
steps/is_last Tensor tf.bool
steps/is_terminal Tensor tf.bool
steps/observation FeaturesDict
steps/observation/object Tensor (14,) tf.float64
steps/observation/robot0_eef_pos Tensor (3,) tf.float64
steps/observation/robot0_eef_quat Tensor (4,) tf.float64
steps/observation/robot0_eef_vel_ang Tensor (3,) tf.float64
steps/observation/robot0_eef_vel_lin Tensor (3,) tf.float64
steps/observation/robot0_gripper_qpos Tensor (2,) tf.float64
steps/observation/robot0_gripper_qvel Tensor (2,) tf.float64
steps/observation/robot0_joint_pos Tensor (7,) tf.float64
steps/observation/robot0_joint_pos_cos Tensor (7,) tf.float64
steps/observation/robot0_joint_pos_sin Tensor (7,) tf.float64
steps/observation/robot0_joint_vel Tensor (7,) tf.float64
steps/reward Tensor tf.float64
steps/states Tensor (71,) tf.float64
train Tensor tf.bool
valid Tensor tf.bool

robomimic_ph/can_image

  • Download size: 1.87 GiB

  • Dataset size: 474.55 MiB

  • Auto-cached (documentation): No

  • Feature structure:

FeaturesDict({
    '20_percent': tf.bool,
    '20_percent_train': tf.bool,
    '20_percent_valid': tf.bool,
    '50_percent': tf.bool,
    '50_percent_train': tf.bool,
    '50_percent_valid': tf.bool,
    'episode_id': tf.string,
    'horizon': tf.int32,
    'steps': Dataset({
        'action': Tensor(shape=(7,), dtype=tf.float64),
        'discount': tf.int32,
        'is_first': tf.bool,
        'is_last': tf.bool,
        'is_terminal': tf.bool,
        'observation': FeaturesDict({
            'agentview_image': Image(shape=(84, 84, 3), dtype=tf.uint8),
            'object': Tensor(shape=(14,), dtype=tf.float64),
            'robot0_eef_pos': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eef_quat': Tensor(shape=(4,), dtype=tf.float64),
            'robot0_eef_vel_ang': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eef_vel_lin': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eye_in_hand_image': Image(shape=(84, 84, 3), dtype=tf.uint8),
            'robot0_gripper_qpos': Tensor(shape=(2,), dtype=tf.float64),
            'robot0_gripper_qvel': Tensor(shape=(2,), dtype=tf.float64),
            'robot0_joint_pos': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_pos_cos': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_pos_sin': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_vel': Tensor(shape=(7,), dtype=tf.float64),
        }),
        'reward': tf.float64,
        'states': Tensor(shape=(71,), dtype=tf.float64),
    }),
    'train': tf.bool,
    'valid': tf.bool,
})
  • Feature documentation:
Feature Class Shape Dtype Description
FeaturesDict
20_percent Tensor tf.bool
20_percent_train Tensor tf.bool
20_percent_valid Tensor tf.bool
50_percent Tensor tf.bool
50_percent_train Tensor tf.bool
50_percent_valid Tensor tf.bool
episode_id Tensor tf.string
horizon Tensor tf.int32
steps Dataset
steps/action Tensor (7,) tf.float64
steps/discount Tensor tf.int32
steps/is_first Tensor tf.bool
steps/is_last Tensor tf.bool
steps/is_terminal Tensor tf.bool
steps/observation FeaturesDict
steps/observation/agentview_image Image (84, 84, 3) tf.uint8
steps/observation/object Tensor (14,) tf.float64
steps/observation/robot0_eef_pos Tensor (3,) tf.float64
steps/observation/robot0_eef_quat Tensor (4,) tf.float64
steps/observation/robot0_eef_vel_ang Tensor (3,) tf.float64
steps/observation/robot0_eef_vel_lin Tensor (3,) tf.float64
steps/observation/robot0_eye_in_hand_image Image (84, 84, 3) tf.uint8
steps/observation/robot0_gripper_qpos Tensor (2,) tf.float64
steps/observation/robot0_gripper_qvel Tensor (2,) tf.float64
steps/observation/robot0_joint_pos Tensor (7,) tf.float64
steps/observation/robot0_joint_pos_cos Tensor (7,) tf.float64
steps/observation/robot0_joint_pos_sin Tensor (7,) tf.float64
steps/observation/robot0_joint_vel Tensor (7,) tf.float64
steps/reward Tensor tf.float64
steps/states Tensor (71,) tf.float64
train Tensor tf.bool
valid Tensor tf.bool

robomimic_ph/square_low_dim

  • Download size: 47.69 MiB

  • Dataset size: 29.91 MiB

  • Auto-cached (documentation): Yes

  • Feature structure:

FeaturesDict({
    '20_percent': tf.bool,
    '20_percent_train': tf.bool,
    '20_percent_valid': tf.bool,
    '50_percent': tf.bool,
    '50_percent_train': tf.bool,
    '50_percent_valid': tf.bool,
    'episode_id': tf.string,
    'horizon': tf.int32,
    'steps': Dataset({
        'action': Tensor(shape=(7,), dtype=tf.float64),
        'discount': tf.int32,
        'is_first': tf.bool,
        'is_last': tf.bool,
        'is_terminal': tf.bool,
        'observation': FeaturesDict({
            'object': Tensor(shape=(14,), dtype=tf.float64),
            'robot0_eef_pos': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eef_quat': Tensor(shape=(4,), dtype=tf.float64),
            'robot0_eef_vel_ang': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eef_vel_lin': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_gripper_qpos': Tensor(shape=(2,), dtype=tf.float64),
            'robot0_gripper_qvel': Tensor(shape=(2,), dtype=tf.float64),
            'robot0_joint_pos': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_pos_cos': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_pos_sin': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_vel': Tensor(shape=(7,), dtype=tf.float64),
        }),
        'reward': tf.float64,
        'states': Tensor(shape=(45,), dtype=tf.float64),
    }),
    'train': tf.bool,
    'valid': tf.bool,
})
  • Feature documentation:
Feature Class Shape Dtype Description
FeaturesDict
20_percent Tensor tf.bool
20_percent_train Tensor tf.bool
20_percent_valid Tensor tf.bool
50_percent Tensor tf.bool
50_percent_train Tensor tf.bool
50_percent_valid Tensor tf.bool
episode_id Tensor tf.string
horizon Tensor tf.int32
steps Dataset
steps/action Tensor (7,) tf.float64
steps/discount Tensor tf.int32
steps/is_first Tensor tf.bool
steps/is_last Tensor tf.bool
steps/is_terminal Tensor tf.bool
steps/observation FeaturesDict
steps/observation/object Tensor (14,) tf.float64
steps/observation/robot0_eef_pos Tensor (3,) tf.float64
steps/observation/robot0_eef_quat Tensor (4,) tf.float64
steps/observation/robot0_eef_vel_ang Tensor (3,) tf.float64
steps/observation/robot0_eef_vel_lin Tensor (3,) tf.float64
steps/observation/robot0_gripper_qpos Tensor (2,) tf.float64
steps/observation/robot0_gripper_qvel Tensor (2,) tf.float64
steps/observation/robot0_joint_pos Tensor (7,) tf.float64
steps/observation/robot0_joint_pos_cos Tensor (7,) tf.float64
steps/observation/robot0_joint_pos_sin Tensor (7,) tf.float64
steps/observation/robot0_joint_vel Tensor (7,) tf.float64
steps/reward Tensor tf.float64
steps/states Tensor (45,) tf.float64
train Tensor tf.bool
valid Tensor tf.bool

robomimic_ph/square_image

  • Download size: 2.42 GiB

  • Dataset size: 401.28 MiB

  • Auto-cached (documentation): No

  • Feature structure:

FeaturesDict({
    '20_percent': tf.bool,
    '20_percent_train': tf.bool,
    '20_percent_valid': tf.bool,
    '50_percent': tf.bool,
    '50_percent_train': tf.bool,
    '50_percent_valid': tf.bool,
    'episode_id': tf.string,
    'horizon': tf.int32,
    'steps': Dataset({
        'action': Tensor(shape=(7,), dtype=tf.float64),
        'discount': tf.int32,
        'is_first': tf.bool,
        'is_last': tf.bool,
        'is_terminal': tf.bool,
        'observation': FeaturesDict({
            'agentview_image': Image(shape=(84, 84, 3), dtype=tf.uint8),
            'object': Tensor(shape=(14,), dtype=tf.float64),
            'robot0_eef_pos': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eef_quat': Tensor(shape=(4,), dtype=tf.float64),
            'robot0_eef_vel_ang': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eef_vel_lin': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eye_in_hand_image': Image(shape=(84, 84, 3), dtype=tf.uint8),
            'robot0_gripper_qpos': Tensor(shape=(2,), dtype=tf.float64),
            'robot0_gripper_qvel': Tensor(shape=(2,), dtype=tf.float64),
            'robot0_joint_pos': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_pos_cos': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_pos_sin': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_vel': Tensor(shape=(7,), dtype=tf.float64),
        }),
        'reward': tf.float64,
        'states': Tensor(shape=(45,), dtype=tf.float64),
    }),
    'train': tf.bool,
    'valid': tf.bool,
})
  • Feature documentation:
Feature Class Shape Dtype Description
FeaturesDict
20_percent Tensor tf.bool
20_percent_train Tensor tf.bool
20_percent_valid Tensor tf.bool
50_percent Tensor tf.bool
50_percent_train Tensor tf.bool
50_percent_valid Tensor tf.bool
episode_id Tensor tf.string
horizon Tensor tf.int32
steps Dataset
steps/action Tensor (7,) tf.float64
steps/discount Tensor tf.int32
steps/is_first Tensor tf.bool
steps/is_last Tensor tf.bool
steps/is_terminal Tensor tf.bool
steps/observation FeaturesDict
steps/observation/agentview_image Image (84, 84, 3) tf.uint8
steps/observation/object Tensor (14,) tf.float64
steps/observation/robot0_eef_pos Tensor (3,) tf.float64
steps/observation/robot0_eef_quat Tensor (4,) tf.float64
steps/observation/robot0_eef_vel_ang Tensor (3,) tf.float64
steps/observation/robot0_eef_vel_lin Tensor (3,) tf.float64
steps/observation/robot0_eye_in_hand_image Image (84, 84, 3) tf.uint8
steps/observation/robot0_gripper_qpos Tensor (2,) tf.float64
steps/observation/robot0_gripper_qvel Tensor (2,) tf.float64
steps/observation/robot0_joint_pos Tensor (7,) tf.float64
steps/observation/robot0_joint_pos_cos Tensor (7,) tf.float64
steps/observation/robot0_joint_pos_sin Tensor (7,) tf.float64
steps/observation/robot0_joint_vel Tensor (7,) tf.float64
steps/reward Tensor tf.float64
steps/states Tensor (45,) tf.float64
train Tensor tf.bool
valid Tensor tf.bool

robomimic_ph/transport_low_dim

  • Download size: 294.70 MiB

  • Dataset size: 208.05 MiB

  • Auto-cached (documentation): Only when shuffle_files=False (train)

  • Feature structure:

FeaturesDict({
    '20_percent': tf.bool,
    '20_percent_train': tf.bool,
    '20_percent_valid': tf.bool,
    '50_percent': tf.bool,
    '50_percent_train': tf.bool,
    '50_percent_valid': tf.bool,
    'episode_id': tf.string,
    'horizon': tf.int32,
    'steps': Dataset({
        'action': Tensor(shape=(14,), dtype=tf.float64),
        'discount': tf.int32,
        'is_first': tf.bool,
        'is_last': tf.bool,
        'is_terminal': tf.bool,
        'observation': FeaturesDict({
            'object': Tensor(shape=(41,), dtype=tf.float64),
            'robot0_eef_pos': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eef_quat': Tensor(shape=(4,), dtype=tf.float64),
            'robot0_eef_vel_ang': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eef_vel_lin': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_gripper_qpos': Tensor(shape=(2,), dtype=tf.float64),
            'robot0_gripper_qvel': Tensor(shape=(2,), dtype=tf.float64),
            'robot0_joint_pos': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_pos_cos': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_pos_sin': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_vel': Tensor(shape=(7,), dtype=tf.float64),
            'robot1_eef_pos': Tensor(shape=(3,), dtype=tf.float64),
            'robot1_eef_quat': Tensor(shape=(4,), dtype=tf.float64),
            'robot1_eef_vel_ang': Tensor(shape=(3,), dtype=tf.float64),
            'robot1_eef_vel_lin': Tensor(shape=(3,), dtype=tf.float64),
            'robot1_gripper_qpos': Tensor(shape=(2,), dtype=tf.float64),
            'robot1_gripper_qvel': Tensor(shape=(2,), dtype=tf.float64),
            'robot1_joint_pos': Tensor(shape=(7,), dtype=tf.float64),
            'robot1_joint_pos_cos': Tensor(shape=(7,), dtype=tf.float64),
            'robot1_joint_pos_sin': Tensor(shape=(7,), dtype=tf.float64),
            'robot1_joint_vel': Tensor(shape=(7,), dtype=tf.float64),
        }),
        'reward': tf.float64,
        'states': Tensor(shape=(115,), dtype=tf.float64),
    }),
    'train': tf.bool,
    'valid': tf.bool,
})
  • Feature documentation:
Feature Class Shape Dtype Description
FeaturesDict
20_percent Tensor tf.bool
20_percent_train Tensor tf.bool
20_percent_valid Tensor tf.bool
50_percent Tensor tf.bool
50_percent_train Tensor tf.bool
50_percent_valid Tensor tf.bool
episode_id Tensor tf.string
horizon Tensor tf.int32
steps Dataset
steps/action Tensor (14,) tf.float64
steps/discount Tensor tf.int32
steps/is_first Tensor tf.bool
steps/is_last Tensor tf.bool
steps/is_terminal Tensor tf.bool
steps/observation FeaturesDict
steps/observation/object Tensor (41,) tf.float64
steps/observation/robot0_eef_pos Tensor (3,) tf.float64
steps/observation/robot0_eef_quat Tensor (4,) tf.float64
steps/observation/robot0_eef_vel_ang Tensor (3,) tf.float64
steps/observation/robot0_eef_vel_lin Tensor (3,) tf.float64
steps/observation/robot0_gripper_qpos Tensor (2,) tf.float64
steps/observation/robot0_gripper_qvel Tensor (2,) tf.float64
steps/observation/robot0_joint_pos Tensor (7,) tf.float64
steps/observation/robot0_joint_pos_cos Tensor (7,) tf.float64
steps/observation/robot0_joint_pos_sin Tensor (7,) tf.float64
steps/observation/robot0_joint_vel Tensor (7,) tf.float64
steps/observation/robot1_eef_pos Tensor (3,) tf.float64
steps/observation/robot1_eef_quat Tensor (4,) tf.float64
steps/observation/robot1_eef_vel_ang Tensor (3,) tf.float64
steps/observation/robot1_eef_vel_lin Tensor (3,) tf.float64
steps/observation/robot1_gripper_qpos Tensor (2,) tf.float64
steps/observation/robot1_gripper_qvel Tensor (2,) tf.float64
steps/observation/robot1_joint_pos Tensor (7,) tf.float64
steps/observation/robot1_joint_pos_cos Tensor (7,) tf.float64
steps/observation/robot1_joint_pos_sin Tensor (7,) tf.float64
steps/observation/robot1_joint_vel Tensor (7,) tf.float64
steps/reward Tensor tf.float64
steps/states Tensor (115,) tf.float64
train Tensor tf.bool
valid Tensor tf.bool

robomimic_ph/transport_image

  • Download size: 15.07 GiB

  • Dataset size: 3.64 GiB

  • Auto-cached (documentation): No

  • Feature structure:

FeaturesDict({
    '20_percent': tf.bool,
    '20_percent_train': tf.bool,
    '20_percent_valid': tf.bool,
    '50_percent': tf.bool,
    '50_percent_train': tf.bool,
    '50_percent_valid': tf.bool,
    'episode_id': tf.string,
    'horizon': tf.int32,
    'steps': Dataset({
        'action': Tensor(shape=(14,), dtype=tf.float64),
        'discount': tf.int32,
        'is_first': tf.bool,
        'is_last': tf.bool,
        'is_terminal': tf.bool,
        'observation': FeaturesDict({
            'object': Tensor(shape=(41,), dtype=tf.float64),
            'robot0_eef_pos': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eef_quat': Tensor(shape=(4,), dtype=tf.float64),
            'robot0_eef_vel_ang': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eef_vel_lin': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eye_in_hand_image': Image(shape=(84, 84, 3), dtype=tf.uint8),
            'robot0_gripper_qpos': Tensor(shape=(2,), dtype=tf.float64),
            'robot0_gripper_qvel': Tensor(shape=(2,), dtype=tf.float64),
            'robot0_joint_pos': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_pos_cos': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_pos_sin': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_vel': Tensor(shape=(7,), dtype=tf.float64),
            'robot1_eef_pos': Tensor(shape=(3,), dtype=tf.float64),
            'robot1_eef_quat': Tensor(shape=(4,), dtype=tf.float64),
            'robot1_eef_vel_ang': Tensor(shape=(3,), dtype=tf.float64),
            'robot1_eef_vel_lin': Tensor(shape=(3,), dtype=tf.float64),
            'robot1_eye_in_hand_image': Image(shape=(84, 84, 3), dtype=tf.uint8),
            'robot1_gripper_qpos': Tensor(shape=(2,), dtype=tf.float64),
            'robot1_gripper_qvel': Tensor(shape=(2,), dtype=tf.float64),
            'robot1_joint_pos': Tensor(shape=(7,), dtype=tf.float64),
            'robot1_joint_pos_cos': Tensor(shape=(7,), dtype=tf.float64),
            'robot1_joint_pos_sin': Tensor(shape=(7,), dtype=tf.float64),
            'robot1_joint_vel': Tensor(shape=(7,), dtype=tf.float64),
            'shouldercamera0_image': Image(shape=(84, 84, 3), dtype=tf.uint8),
            'shouldercamera1_image': Image(shape=(84, 84, 3), dtype=tf.uint8),
        }),
        'reward': tf.float64,
        'states': Tensor(shape=(115,), dtype=tf.float64),
    }),
    'train': tf.bool,
    'valid': tf.bool,
})
  • Feature documentation:
Feature Class Shape Dtype Description
FeaturesDict
20_percent Tensor tf.bool
20_percent_train Tensor tf.bool
20_percent_valid Tensor tf.bool
50_percent Tensor tf.bool
50_percent_train Tensor tf.bool
50_percent_valid Tensor tf.bool
episode_id Tensor tf.string
horizon Tensor tf.int32
steps Dataset
steps/action Tensor (14,) tf.float64
steps/discount Tensor tf.int32
steps/is_first Tensor tf.bool
steps/is_last Tensor tf.bool
steps/is_terminal Tensor tf.bool
steps/observation FeaturesDict
steps/observation/object Tensor (41,) tf.float64
steps/observation/robot0_eef_pos Tensor (3,) tf.float64
steps/observation/robot0_eef_quat Tensor (4,) tf.float64
steps/observation/robot0_eef_vel_ang Tensor (3,) tf.float64
steps/observation/robot0_eef_vel_lin Tensor (3,) tf.float64
steps/observation/robot0_eye_in_hand_image Image (84, 84, 3) tf.uint8
steps/observation/robot0_gripper_qpos Tensor (2,) tf.float64
steps/observation/robot0_gripper_qvel Tensor (2,) tf.float64
steps/observation/robot0_joint_pos Tensor (7,) tf.float64
steps/observation/robot0_joint_pos_cos Tensor (7,) tf.float64
steps/observation/robot0_joint_pos_sin Tensor (7,) tf.float64
steps/observation/robot0_joint_vel Tensor (7,) tf.float64
steps/observation/robot1_eef_pos Tensor (3,) tf.float64
steps/observation/robot1_eef_quat Tensor (4,) tf.float64
steps/observation/robot1_eef_vel_ang Tensor (3,) tf.float64
steps/observation/robot1_eef_vel_lin Tensor (3,) tf.float64
steps/observation/robot1_eye_in_hand_image Image (84, 84, 3) tf.uint8
steps/observation/robot1_gripper_qpos Tensor (2,) tf.float64
steps/observation/robot1_gripper_qvel Tensor (2,) tf.float64
steps/observation/robot1_joint_pos Tensor (7,) tf.float64
steps/observation/robot1_joint_pos_cos Tensor (7,) tf.float64
steps/observation/robot1_joint_pos_sin Tensor (7,) tf.float64
steps/observation/robot1_joint_vel Tensor (7,) tf.float64
steps/observation/shouldercamera0_image Image (84, 84, 3) tf.uint8
steps/observation/shouldercamera1_image Image (84, 84, 3) tf.uint8
steps/reward Tensor tf.float64
steps/states Tensor (115,) tf.float64
train Tensor tf.bool
valid Tensor tf.bool

robomimic_ph/tool_hang_low_dim

  • Download size: 192.29 MiB

  • Dataset size: 121.77 MiB

  • Auto-cached (documentation): Yes

  • Feature structure:

FeaturesDict({
    'episode_id': tf.string,
    'horizon': tf.int32,
    'steps': Dataset({
        'action': Tensor(shape=(7,), dtype=tf.float64),
        'discount': tf.int32,
        'is_first': tf.bool,
        'is_last': tf.bool,
        'is_terminal': tf.bool,
        'observation': FeaturesDict({
            'object': Tensor(shape=(44,), dtype=tf.float64),
            'robot0_eef_pos': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eef_quat': Tensor(shape=(4,), dtype=tf.float64),
            'robot0_eef_vel_ang': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eef_vel_lin': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_gripper_qpos': Tensor(shape=(2,), dtype=tf.float64),
            'robot0_gripper_qvel': Tensor(shape=(2,), dtype=tf.float64),
            'robot0_joint_pos': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_pos_cos': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_pos_sin': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_vel': Tensor(shape=(7,), dtype=tf.float64),
        }),
        'reward': tf.float64,
        'states': Tensor(shape=(58,), dtype=tf.float64),
    }),
    'train': tf.bool,
    'valid': tf.bool,
})
  • Feature documentation:
Feature Class Shape Dtype Description
FeaturesDict
episode_id Tensor tf.string
horizon Tensor tf.int32
steps Dataset
steps/action Tensor (7,) tf.float64
steps/discount Tensor tf.int32
steps/is_first Tensor tf.bool
steps/is_last Tensor tf.bool
steps/is_terminal Tensor tf.bool
steps/observation FeaturesDict
steps/observation/object Tensor (44,) tf.float64
steps/observation/robot0_eef_pos Tensor (3,) tf.float64
steps/observation/robot0_eef_quat Tensor (4,) tf.float64
steps/observation/robot0_eef_vel_ang Tensor (3,) tf.float64
steps/observation/robot0_eef_vel_lin Tensor (3,) tf.float64
steps/observation/robot0_gripper_qpos Tensor (2,) tf.float64
steps/observation/robot0_gripper_qvel Tensor (2,) tf.float64
steps/observation/robot0_joint_pos Tensor (7,) tf.float64
steps/observation/robot0_joint_pos_cos Tensor (7,) tf.float64
steps/observation/robot0_joint_pos_sin Tensor (7,) tf.float64
steps/observation/robot0_joint_vel Tensor (7,) tf.float64
steps/reward Tensor tf.float64
steps/states Tensor (58,) tf.float64
train Tensor tf.bool
valid Tensor tf.bool

robomimic_ph/tool_hang_image

  • Download size: 61.96 GiB

  • Dataset size: 9.10 GiB

  • Auto-cached (documentation): No

  • Feature structure:

FeaturesDict({
    'episode_id': tf.string,
    'horizon': tf.int32,
    'steps': Dataset({
        'action': Tensor(shape=(7,), dtype=tf.float64),
        'discount': tf.int32,
        'is_first': tf.bool,
        'is_last': tf.bool,
        'is_terminal': tf.bool,
        'observation': FeaturesDict({
            'object': Tensor(shape=(44,), dtype=tf.float64),
            'robot0_eef_pos': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eef_quat': Tensor(shape=(4,), dtype=tf.float64),
            'robot0_eef_vel_ang': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eef_vel_lin': Tensor(shape=(3,), dtype=tf.float64),
            'robot0_eye_in_hand_image': Image(shape=(240, 240, 3), dtype=tf.uint8),
            'robot0_gripper_qpos': Tensor(shape=(2,), dtype=tf.float64),
            'robot0_gripper_qvel': Tensor(shape=(2,), dtype=tf.float64),
            'robot0_joint_pos': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_pos_cos': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_pos_sin': Tensor(shape=(7,), dtype=tf.float64),
            'robot0_joint_vel': Tensor(shape=(7,), dtype=tf.float64),
            'sideview_image': Image(shape=(240, 240, 3), dtype=tf.uint8),
        }),
        'reward': tf.float64,
        'states': Tensor(shape=(58,), dtype=tf.float64),
    }),
    'train': tf.bool,
    'valid': tf.bool,
})
  • Feature documentation:
Feature Class Shape Dtype Description
FeaturesDict
episode_id Tensor tf.string
horizon Tensor tf.int32
steps Dataset
steps/action Tensor (7,) tf.float64
steps/discount Tensor tf.int32
steps/is_first Tensor tf.bool
steps/is_last Tensor tf.bool
steps/is_terminal Tensor tf.bool
steps/observation FeaturesDict
steps/observation/object Tensor (44,) tf.float64
steps/observation/robot0_eef_pos Tensor (3,) tf.float64
steps/observation/robot0_eef_quat Tensor (4,) tf.float64
steps/observation/robot0_eef_vel_ang Tensor (3,) tf.float64
steps/observation/robot0_eef_vel_lin Tensor (3,) tf.float64
steps/observation/robot0_eye_in_hand_image Image (240, 240, 3) tf.uint8
steps/observation/robot0_gripper_qpos Tensor (2,) tf.float64
steps/observation/robot0_gripper_qvel Tensor (2,) tf.float64
steps/observation/robot0_joint_pos Tensor (7,) tf.float64
steps/observation/robot0_joint_pos_cos Tensor (7,) tf.float64
steps/observation/robot0_joint_pos_sin Tensor (7,) tf.float64
steps/observation/robot0_joint_vel Tensor (7,) tf.float64
steps/observation/sideview_image Image (240, 240, 3) tf.uint8
steps/reward Tensor tf.float64
steps/states Tensor (58,) tf.float64
train Tensor tf.bool
valid Tensor tf.bool