Announcing the TensorFlow Dev Summit 2020 Learn more

tfds.features.FeaturesDict

View source on GitHub

Class FeaturesDict

Composite FeatureConnector; each feature in dict has its own connector.

The encode/decode method of the spec feature will recursively encode/decode every sub-connector given on the constructor. Other features can inherit from this class and call super() in order to get nested container.

Example:

For DatasetInfo:

features = tfds.features.FeaturesDict({
    'input': tfds.features.Image(),
    'output': tf.int32,
})

At generation time:

for image, label in generate_examples:
  yield {
      'input': image,
      'output': label
  }

At tf.data.Dataset() time:

for example in tfds.load(...):
  tf_input = example['input']
  tf_output = example['output']

For nested features, the FeaturesDict will internally flatten the keys for the features and the conversion to tf.train.Example. Indeed, the tf.train.Example proto do not support nested feature, while tf.data.Dataset does. But internal transformation should be invisible to the user.

Example:

tfds.features.FeaturesDict({
    'input': tf.int32,
    'target': {
        'height': tf.int32,
        'width': tf.int32,
    },
})

Will internally store the data as:

{
    'input': tf.io.FixedLenFeature(shape=(), dtype=tf.int32),
    'target/height': tf.io.FixedLenFeature(shape=(), dtype=tf.int32),
    'target/width': tf.io.FixedLenFeature(shape=(), dtype=tf.int32),
}

__init__

View source

__init__(feature_dict)

Initialize the features.

Args:

feature_dict (dict): Dictionary containing the feature connectors of a example. The keys should correspond to the data dict as returned by tf.data.Dataset(). Types (tf.int32,...) and dicts will automatically be converted into FeatureConnector.

Raises:

  • ValueError: If one of the given features is not recognized

Properties

dtype

Return the dtype (or dict of dtype) of this FeatureConnector.

shape

Return the shape (or dict of shape) of this FeatureConnector.

Methods

__contains__

View source

__contains__(k)

__getitem__

View source

__getitem__(key)

Return the feature associated with the key.

__iter__

View source

__iter__()

__len__

View source

__len__()

decode_batch_example

View source

decode_batch_example(tfexample_data)

Decode multiple features batched in a single tf.Tensor.

This function is used to decode features wrapped in tfds.features.Sequence(). By default, this function apply decode_example on each individual elements using tf.map_fn. However, for optimization, features can overwrite this method to apply a custom batch decoding.

Args:

  • tfexample_data: Same tf.Tensor inputs as decode_example, but with and additional first dimension for the sequence length.

Returns:

  • tensor_data: Tensor or dictionary of tensor, output of the tf.data.Dataset object

decode_example

decode_example(
    *args,
    **kwargs
)

Decode the serialize examples.

Args:

  • serialized_example: Nested dict of tf.Tensor
  • decoders: Nested dict of Decoder objects which allow to customize the decoding. The structure should match the feature structure, but only customized feature keys need to be present. See the guide for more info.

Returns:

  • example: Nested dict containing the decoded nested examples.

decode_ragged_example

View source

decode_ragged_example(tfexample_data)

Decode nested features from a tf.RaggedTensor.

This function is used to decode features wrapped in nested tfds.features.Sequence(). By default, this function apply decode_batch_example on the flat values of the ragged tensor. For optimization, features can overwrite this method to apply a custom batch decoding.

Args:

  • tfexample_data: tf.RaggedTensor inputs containing the nested encoded examples.

Returns:

  • tensor_data: The decoded tf.RaggedTensor or dictionary of tensor, output of the tf.data.Dataset object

encode_example

View source

encode_example(example_dict)

See base class for details.

get_serialized_info

View source

get_serialized_info()

See base class for details.

get_tensor_info

View source

get_tensor_info()

See base class for details.

items

View source

items()

keys

View source

keys()

load_metadata

View source

load_metadata(
    data_dir,
    feature_name=None
)

See base class for details.

save_metadata

View source

save_metadata(
    data_dir,
    feature_name=None
)

See base class for details.

values

View source

values()