tfds.features.Video

View source on GitHub

Class Video

FeatureConnector for videos, encoding frames individually on disk.

Inherits From: Sequence

Video: The image connector accepts as input a 4 dimensional tf.uint8 array representing a video, a sequence of paths to encoded frames, or a path or a file object that can be decoded with ffmpeg. Note that not all formats in ffmpeg support reading from pipes, so providing a file object might fail. Furthermore, if a path is given that is not on the local file system, we first copy it to a temporary local file before passing it to ffmpeg.

Output:

• video: tf.Tensor of type tf.uint8 and shape [num_frames, height, width, channels], where channels must be 1 or 3

Example:

• In the DatasetInfo object:

features=features.FeatureDict({
    'video': features.Video(shape=(None, 64, 64, 3)),
})

• During generation, you can use any of:

yield {
    'video': np.ones(shape=(128, 64, 64, 3), dtype=np.uint8),
}

or list of frames:

yield {
    'video': ['path/to/frame001.png', 'path/to/frame002.png'],
}

or path to video:

yield {
    'video': '/path/to/video.avi',
}

or file object:

yield {
    'video': tf.io.gfile.GFile('/complex/path/video.avi'),
}

__init__

View source

__init__(
    shape,
    encoding_format='png',
    ffmpeg_extra_args=()
)

Initializes the connector.

Args:

• shape: tuple of ints, the shape of the video (num_frames, height, width, channels), where channels is 1 or 3.
• encoding_format: The video is stored as a sequence of encoded images. You can use any encoding format supported by image_feature.Feature.
• ffmpeg_extra_args: A sequence of additional args to be passed to the ffmpeg binary. Specifically, ffmpeg will be called as: ffmpeg -i <input_file> <ffmpeg_extra_args> %010d.<encoding_format>

Raises:

• ValueError: If the shape is invalid

Properties

dtype

Return the dtype (or dict of dtype) of this FeatureConnector.

feature

The inner feature.

shape

Return the shape (or dict of shape) of this FeatureConnector.

Methods

__getitem__

View source

__getitem__(key)

Convenience method to access the underlying features.

decode_batch_example

View source

decode_batch_example(tfexample_data)

Decode multiple features batched in a single tf.Tensor.

This function is used to decode features wrapped in tfds.features.Sequence(). By default, this function apply decode_example on each individual elements using tf.map_fn. However, for optimization, features can overwrite this method to apply a custom batch decoding.

Args:

• tfexample_data: Same tf.Tensor inputs as decode_example, but with and additional first dimension for the sequence length.

Returns:

• tensor_data: Tensor or dictionary of tensor, output of the tf.data.Dataset object

decode_example

decode_example(
    *args,
    **kwargs
)

Decode the serialize examples.

Args:

• serialized_example: Nested dict of tf.Tensor
• decoders: Nested dict of Decoder objects which allow to customize the decoding. The structure should match the feature structure, but only customized feature keys need to be present. See the guide for more info.

Returns:

• example: Nested dict containing the decoded nested examples.

decode_ragged_example

View source

decode_ragged_example(tfexample_data)

Decode nested features from a tf.RaggedTensor.

This function is used to decode features wrapped in nested tfds.features.Sequence(). By default, this function apply decode_batch_example on the flat values of the ragged tensor. For optimization, features can overwrite this method to apply a custom batch decoding.

Args:

• tfexample_data: tf.RaggedTensor inputs containing the nested encoded examples.

Returns:

• tensor_data: The decoded tf.RaggedTensor or dictionary of tensor, output of the tf.data.Dataset object

encode_example

View source

encode_example(video_or_path_or_fobj)

Converts the given image into a dict convertible to tf example.

get_serialized_info

View source

get_serialized_info()

See base class for details.

get_tensor_info

View source

get_tensor_info()

See base class for details.

load_metadata

View source

load_metadata(
    *args,
    **kwargs
)

See base class for details.

save_metadata

View source

save_metadata(
    *args,
    **kwargs
)

See base class for details.