bc_z

  • Description:

Teleoped Google robot doing mostly pick-place from a table

Split Examples
'train' 39,350
'val' 3,914
  • Feature structure:
FeaturesDict({
    'steps': Dataset({
        'action': FeaturesDict({
            'future/axis_angle_residual': Tensor(shape=(30,), dtype=float32),
            'future/target_close': Tensor(shape=(10,), dtype=int64),
            'future/xyz_residual': Tensor(shape=(30,), dtype=float32),
        }),
        'is_first': bool,
        'is_last': bool,
        'is_terminal': bool,
        'observation': FeaturesDict({
            'episode_success': float32,
            'image': Image(shape=(171, 213, 3), dtype=uint8),
            'natural_language_embedding': Tensor(shape=(512,), dtype=float32),
            'natural_language_instruction': string,
            'present/autonomous': int64,
            'present/axis_angle': Tensor(shape=(3,), dtype=float32),
            'present/intervention': int64,
            'present/sensed_close': Tensor(shape=(1,), dtype=float32),
            'present/xyz': Tensor(shape=(3,), dtype=float32),
            'sequence_length': int64,
        }),
        'reward': Scalar(shape=(), dtype=float32),
    }),
})
  • Feature documentation:
Feature Class Shape Dtype Description
FeaturesDict
steps Dataset
steps/action FeaturesDict
steps/action/future/axis_angle_residual Tensor (30,) float32 The next 10 actions for the rotation. Each action is a 3D delta to add to the current axis angle.
steps/action/future/target_close Tensor (10,) int64 The next 10 actions for the gripper. Each action is the value the gripper closure should be changed to (notably it is not a delta.)
steps/action/future/xyz_residual Tensor (30,) float32 The next 10 actions for the positions. Each action is a 3D delta to add to current position.
steps/is_first Tensor bool
steps/is_last Tensor bool
steps/is_terminal Tensor bool
steps/observation FeaturesDict
steps/observation/episode_success Tensor float32 A 0-1 success label
steps/observation/image Image (171, 213, 3) uint8 Camera image of the robot, downsampled 3x
steps/observation/natural_language_embedding Tensor (512,) float32 An embedding of the task via Universal Sentence Encoder (https://tfhub.dev/google/universal-sentence-encoder/4)
steps/observation/natural_language_instruction Tensor string The task the robot was asked to do.
steps/observation/present/autonomous Tensor int64 Episodes are collected via DAgger. This is a 0/1 label for whether the action is from the policy or the teleoperator. 1 = from policy.
steps/observation/present/axis_angle Tensor (3,) float32 The current rotation of the end effector in axis-angle representation.
steps/observation/present/intervention Tensor int64 Episodes are collected via DAgger. This is a 0/1 label for whether the action is from the policy or the teleoperator. 1 = from teleoperator. This is exactly the opposite of present/autonomous
steps/observation/present/sensed_close Tensor (1,) float32 How much the gripper is currently closed. Scaled from 0 to 1, but not all values from 0 to 1 are reachable. The range in the data is about 0.2 to 1
steps/observation/present/xyz Tensor (3,) float32 The current position of the end effector in axis-angle representation, in robot frame
steps/observation/sequence_length Tensor int64 Length of the episode
steps/reward Scalar float32
  • Citation:
@inproceedings{jang2021bc,
title={ {BC}-Z: Zero-Shot Task Generalization with Robotic Imitation Learning},
author={Eric Jang and Alex Irpan and Mohi Khansari and Daniel Kappler and Frederik Ebert and Corey Lynch and Sergey Levine and Chelsea Finn},
booktitle={5th Annual Conference on Robot Learning},
year={2021},
url={https://openreview.net/forum?id=8kbp23tSGYv} }