tfg.datasets.features.Camera

FeatureConnector for camera calibration (extrinsic and intrinsic).

During _generate_examples, the feature connector accepts as input:

  • parameter_dict: A dictionary containing the extrinsic and instrinsic parameters of the camera as:
    • 'pose': Dictionary containing
      • Either 3x3 rotation matrix and translation vector: { 'R': A float32 tensor with shape [3, 3] denoting the 3D rotation matrix. 't': A float32 tensor with shape [3,] denoting the translation vector. } OR
      • look_at, position and up-vector: { 'look_at': float32 vector of shape (3,). 'position': float32 vector of shape (3,). 'up': float32 vector of shape (3,). }
    • 'f': focal length of the camera in pixel (either single float32 value or tuple of float32 as (f_x, f_y).
    • 'optical_center': Optical center of the camera in pixel coordinates as tuple (c_x, c_y) Optional parameters:
    • 'skew': float32 denoting the skew of the camera axes.
    • 'aspect_ratio': float32 denoting the aspect_ratio, if single fixed focal length is provided.

A dictionary containing:

  • 'pose': A tensorflow_graphics.datasets.features.Pose FeatureConnector representing the 3D pose of the camera.

  • 'intrinsics': A float32 tensor with shape [3,3] denoting the intrinsic matrix.

Default values for skew (s) and aspect_ratio(a) are 0 and 1, respectively.

Full calibration matrix: K = [[ f_x, s, c_x ], [ 0, f_y, c_y ], [ 0, 0, 1 ]]

With same focal length: K = [[ f, s, c_x ], [ 0, af, c_y ], [ 0, 0, 1 ]]

feature_dict (dict): Dictionary containing the feature connectors of a example. The keys should correspond to the data dict as returned by tf.data.Dataset(). Types (tf.int32,...) and dicts will automatically be converted into FeatureConnector.
doc Documentation of this feature (e.g. description).

ValueError If one of the given features is not recognized

doc

dtype Return the dtype (or dict of dtype) of this FeatureConnector.
numpy_dtype

shape Return the shape (or dict of shape) of this FeatureConnector.
tf_example_spec Returns the tf.Example proto structure.

Methods

catalog_documentation

Returns the feature documentation to be shown in the catalog.

cls_from_name

Returns the feature class for the given Python class.

decode_batch_example

Decode multiple features batched in a single tf.Tensor.

This function is used to decode features wrapped in tfds.features.Sequence(). By default, this function apply decode_example on each individual elements using tf.map_fn. However, for optimization, features can overwrite this method to apply a custom batch decoding.

Args
tfexample_data Same tf.Tensor inputs as decode_example, but with and additional first dimension for the sequence length.

Returns
tensor_data Tensor or dictionary of tensor, output of the tf.data.Dataset object

decode_example

Decode the serialize examples.

Args
serialized_example Nested dict of tf.Tensor
decoders Nested dict of Decoder objects which allow to customize the decoding. The structure should match the feature structure, but only customized feature keys need to be present. See the guide for more info.

Returns
example Nested dict containing the decoded nested examples.

decode_ragged_example

Decode nested features from a tf.RaggedTensor.

This function is used to decode features wrapped in nested tfds.features.Sequence(). By default, this function apply decode_batch_example on the flat values of the ragged tensor. For optimization, features can overwrite this method to apply a custom batch decoding.

Args
tfexample_data tf.RaggedTensor inputs containing the nested encoded examples.

Returns
tensor_data The decoded tf.RaggedTensor or dictionary of tensor, output of the tf.data.Dataset object

deserialize_example

Decodes the tf.train.Example data into tf.Tensor.

See serialize_example to encode the data into proto.

Args
serialized_example The tensor-like object containing the serialized tf.train.Example proto.
decoders Eventual decoders to apply (see documentation)

Returns
The decoded features tensors.

encode_example

View source

Convert the given parameters into a dict convertible to tf example.

from_config

Reconstructs the FeatureConnector from the config file.

Usage:

features = FeatureConnector.from_config('path/to/dir')

Args
root_dir Directory containing the features.json file.

Returns
The reconstructed feature instance.

from_json

FeatureConnector factory.

This function should be called from the tfds.features.FeatureConnector base class. Subclass should implement the from_json_content.

Example:

feature = tfds.features.FeatureConnector.from_json(
    {'type': 'Image', 'content': {'shape': [32, 32, 3], 'dtype': 'uint8'}}
)
assert isinstance(feature, tfds.features.Image)

Args
value dict(type=, content=) containing the feature to restore. Match dict returned by to_json.

Returns
The reconstructed FeatureConnector.

from_json_content

View source

FeatureConnector factory (to overwrite).

Subclasses should overwrite this method. This method is used when importing the feature connector from the config.

This function should not be called directly. FeatureConnector.from_json should be called instead.

See existing FeatureConnectors for implementation examples.

Args
value FeatureConnector information represented as either Json or a Feature proto. The content must match what is returned by to_json_content.
doc Documentation of this feature (e.g. description).

Returns
The reconstructed FeatureConnector.

from_proto

Instantiates a feature from its proto representation.

get_serialized_info

See base class for details.

get_tensor_info

See base class for details.

get_tensor_spec

Returns the tf.TensorSpec of this feature (not the element spec!).

Note that the output of this method may not correspond to the element spec of the dataset. For example, currently this method does not support RaggedTensorSpec.

items

keys

load_metadata

See base class for details.

repr_html

Returns the HTML str representation of the object.

repr_html_batch

Returns the HTML str representation of the object (Sequence).

repr_html_ragged

Returns the HTML str representation of the object (Nested sequence).

save_config

Exports the FeatureConnector to a file.

Args
root_dir path/to/dir containing the features.json

save_metadata

See base class for details.

serialize_example

Encodes nested data values into tf.train.Example bytes.

See deserialize_example to decode the proto into tf.Tensor.

Args
example_data Example data to encode (numpy-like nested dict)

Returns
The serialized tf.train.Example.

to_json

Exports the FeatureConnector to Json.

Each feature is serialized as a dict(type=..., content=...).

  • type: The cannonical name of the feature (module.FeatureName).
  • content: is specific to each feature connector and defined in to_json_content. Can contain nested sub-features (like for tfds.features.FeaturesDict and tfds.features.Sequence).

For example:

tfds.features.FeaturesDict({
    'input': tfds.features.Image(),
    'target': tfds.features.ClassLabel(num_classes=10),
})

Is serialized as:

{
    "type": "tensorflow_datasets.core.features.features_dict.FeaturesDict",
    "content": {
        "input": {
            "type": "tensorflow_datasets.core.features.image_feature.Image",
            "content": {
                "shape": [null, null, 3],
                "dtype": "uint8",
                "encoding_format": "png"
            }
        },
        "target": {
            "type":
            "tensorflow_datasets.core.features.class_label_feature.ClassLabel",
            "content": {
              "num_classes": 10
            }
        }
    }
}

Returns
A dict(type=, content=). Will be forwarded to from_json when reconstructing the feature.

to_json_content

View source

FeatureConnector factory (to overwrite).

This function should be overwritten by the subclass to allow re-importing the feature connector from the config. See existing FeatureConnector for example of implementation.

Returns
The FeatureConnector metadata in either a dict, or a Feature proto. This output is used in from_json_content when reconstructing the feature.

to_proto

Exports the FeatureConnector to the Feature proto.

For features that have a specific schema defined in a proto, this function needs to be overriden. If there's no specific proto schema, then the feature will be represented using JSON.

Returns
The feature proto describing this feature.

values

__contains__

__getitem__

Return the feature associated with the key.

__iter__

__len__

ALIASES []