Missed TensorFlow Dev Summit? Check out the video playlist. Watch recordings

tf_agents.networks.value_rnn_network.ValueRnnNetwork

View source on GitHub

Recurrent value network. Reduces to 1 value output per batch item.

Inherits From: Network

tf_agents.networks.value_rnn_network.ValueRnnNetwork(
    input_tensor_spec, preprocessing_layers=None, preprocessing_combiner=None,
    conv_layer_params=None, input_fc_layer_params=(75, 40),
    input_dropout_layer_params=None, lstm_size=(40,), output_fc_layer_params=(75,
    40), activation_fn=tf.keras.activations.relu, dtype=tf.float32,
    name='ValueRnnNetwork'
)

Args:

  • input_tensor_spec: A nest of tensor_spec.TensorSpec representing the input observations.
  • preprocessing_layers: (Optional.) A nest of tf.keras.layers.Layer representing preprocessing for the different observations. All of these layers must not be already built. For more details see the documentation of networks.EncodingNetwork.
  • preprocessing_combiner: (Optional.) A keras layer that takes a flat list of tensors and combines them. Good options include tf.keras.layers.Add and tf.keras.layers.Concatenate(axis=-1). This layer must not be already built. For more details see the documentation of networks.EncodingNetwork.
  • conv_layer_params: Optional list of convolution layers parameters, where each item is a length-three tuple indicating (filters, kernel_size, stride).
  • input_fc_layer_params: Optional list of fully_connected parameters, where each item is the number of units in the layer. This is applied before the LSTM cell.
  • input_dropout_layer_params: Optional list of dropout layer parameters, where each item is the fraction of input units to drop. The dropout layers are interleaved with the fully connected layers; there is a dropout layer after each fully connected layer, except if the entry in the list is None. This list must have the same length of input_fc_layer_params, or be None.
  • lstm_size: An iterable of ints specifying the LSTM cell sizes to use.
  • output_fc_layer_params: Optional list of fully_connected parameters, where each item is the number of units in the layer. This is applied after the LSTM cell.
  • activation_fn: Activation function, e.g. tf.keras.activations.relu,.
  • dtype: The dtype to use by the convolution, LSTM, and fully connected layers.
  • name: A string representing name of the network.

Attributes:

  • activity_regularizer: Optional regularizer function for the output of this layer.
  • dtype
  • dynamic
  • input: Retrieves the input tensor(s) of a layer.

    Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer.

  • input_mask: Retrieves the input mask tensor(s) of a layer.

    Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

  • input_shape: Retrieves the input shape(s) of a layer.

    Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer, or if all inputs have the same shape.

  • input_spec: Gets the network's input specs.

  • input_tensor_spec: Returns the spec of the input to the network of type InputSpec.

  • layers

  • losses: Losses which are associated with this Layer.

    Variable regularization tensors are created when this property is accessed, so it is eager safe: accessing losses under a tf.GradientTape will propagate gradients back to the corresponding variables.

  • metrics

  • name: Returns the name of this module as passed or determined in the ctor.

    NOTE: This is not the same as the self.name_scope.name which includes parent module names.

  • name_scope: Returns a tf.name_scope instance for this class.

  • non_trainable_variables

  • non_trainable_weights

  • output: Retrieves the output tensor(s) of a layer.

    Only applicable if the layer has exactly one output, i.e. if it is connected to one incoming layer.

  • output_mask: Retrieves the output mask tensor(s) of a layer.

    Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.

  • output_shape: Retrieves the output shape(s) of a layer.

    Only applicable if the layer has one output, or if all outputs have the same shape.

  • state_spec

  • state_updates: Returns the updates from all layers that are stateful.

    This is useful for separating training updates and state updates, e.g. when we need to update a layer's internal state during prediction.

  • stateful

  • submodules: Sequence of all sub-modules.

    Submodules are modules which are properties of this module, or found as properties of modules which are properties of this module (and so on).

a = tf.Module()
b = tf.Module()
c = tf.Module()
a.b = b
b.c = c
assert list(a.submodules) == [b, c]
assert list(b.submodules) == [c]
assert list(c.submodules) == []
  • trainable
  • trainable_variables: Sequence of trainable variables owned by this module and its submodules.

  • trainable_weights

  • updates

  • variables: Returns the list of all layer variables/weights.

    Alias of self.weights.

  • weights: Returns the list of all layer variables/weights.

Methods

__call__

View source

__call__(
    inputs, *args, **kwargs
)

A wrapper around Network.call.

A typical call method in a class subclassing Network looks like this:

def call(self,
         observation,
         step_type=None,
         network_state=(),
         training=False):
    ...
    return outputs, new_network_state

In this case, we will validate the first argument (observation) against self.input_tensor_spec.

If a network_state kwarg is given it is also validated against self.state_spec. Similarly, the return value of the call method is expected to be a tuple/list with 2 values: (output, new_state); we validate new_state against self.state_spec.

Args:

  • inputs: The inputs to self.call, matching self.input_state_spec.
  • *args: Additional arguments to self.call.
  • **kwargs: Additional keyword arguments to self.call.

Returns:

A tuple (outputs, new_network_state).

build

build(
    input_shape
)

Builds the model based on input shapes received.

This is to be used for subclassed models, which do not know at instantiation time what their inputs look like.

This method only exists for users who want to call model.build() in a standalone way (as a substitute for calling the model on real data to build it). It will never be called by the framework (and thus it will never throw unexpected errors in an unrelated workflow).

Args:

  • input_shape: Single tuple, TensorShape, or list of shapes, where shapes are tuples, integers, or TensorShapes.

Raises:

  • ValueError: 1. In case of invalid user-provided data (not of type tuple, list, or TensorShape).
    1. If the model requires call arguments that are agnostic to the input shapes (positional or kwarg in call signature).
    2. If not all layers were properly built.
    3. If float type inputs are not supported within the layers.

In each of these cases, the user should build their model by calling it on real tensor data.

compute_mask

compute_mask(
    inputs, mask
)

Computes an output mask tensor.

Arguments:

  • inputs: Tensor or list of tensors.
  • mask: Tensor or list of tensors.

Returns:

None or a tensor (or list of tensors, one per output tensor of the layer).

compute_output_shape

compute_output_shape(
    input_shape
)

Computes the output shape of the layer.

If the layer has not been built, this method will call build on the layer. This assumes that the layer will later be used with inputs that match the input shape provided here.

Arguments:

  • input_shape: Shape tuple (tuple of integers) or list of shape tuples (one per output tensor of the layer). Shape tuples can include None for free dimensions, instead of an integer.

Returns:

An input shape tuple.

copy

View source

copy(
    **kwargs
)

Create a shallow copy of this network.

NOTE Network layer weights are never copied. This method recreates the Network instance with the same arguments it was initialized with (excepting any new kwargs).

Args:

  • **kwargs: Args to override when recreating this network. Commonly overridden args include 'name'.

Returns:

A shallow copy of this network.

count_params

count_params()

Count the total number of scalars composing the weights.

Returns:

An integer count.

Raises:

  • ValueError: if the layer isn't yet built (in which case its weights aren't yet defined).

create_variables

View source

create_variables(
    **kwargs
)

from_config

@classmethod
from_config(
    cls, config, custom_objects=None
)

Instantiates a Model from its config (output of get_config()).

Arguments:

  • config: Model config dictionary.
  • custom_objects: Optional dictionary mapping names (strings) to custom classes or functions to be considered during deserialization.

Returns:

A model instance.

Raises:

  • ValueError: In case of improperly formatted config dict.

get_config

get_config()

Returns the config of the layer.

A layer config is a Python dictionary (serializable) containing the configuration of a layer. The same layer can be reinstantiated later (without its trained weights) from this configuration.

The config of a layer does not include connectivity information, nor the layer class name. These are handled by Network (one layer of abstraction above).

Returns:

Python dictionary.

get_initial_state

View source

get_initial_state(
    batch_size=None
)

Returns an initial state usable by the network.

Args:

  • batch_size: Tensor or constant: size of the batch dimension. Can be None in which case not dimensions gets added.

Returns:

A nested object of type self.state_spec containing properly initialized Tensors.

get_input_at

get_input_at(
    node_index
)

Retrieves the input tensor(s) of a layer at a given node.

Arguments:

  • node_index: Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple inputs).

Raises:

  • RuntimeError: If called in Eager mode.

get_input_mask_at

get_input_mask_at(
    node_index
)

Retrieves the input mask tensor(s) of a layer at a given node.

Arguments:

  • node_index: Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple inputs).

get_input_shape_at

get_input_shape_at(
    node_index
)

Retrieves the input shape(s) of a layer at a given node.

Arguments:

  • node_index: Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple inputs).

Raises:

  • RuntimeError: If called in Eager mode.

get_layer

get_layer(
    name=None, index=None
)

Retrieves a layer based on either its name (unique) or index.

If name and index are both provided, index will take precedence. Indices are based on order of horizontal graph traversal (bottom-up).

Arguments:

  • name: String, name of layer.
  • index: Integer, index of layer.

Returns:

A layer instance.

Raises:

  • ValueError: In case of invalid layer name or index.

get_losses_for

get_losses_for(
    inputs
)

Retrieves losses relevant to a specific set of inputs.

Arguments:

  • inputs: Input tensor or list/tuple of input tensors.

Returns:

List of loss tensors of the layer that depend on inputs.

get_output_at

get_output_at(
    node_index
)

Retrieves the output tensor(s) of a layer at a given node.

Arguments:

  • node_index: Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A tensor (or list of tensors if the layer has multiple outputs).

Raises:

  • RuntimeError: If called in Eager mode.

get_output_mask_at

get_output_mask_at(
    node_index
)

Retrieves the output mask tensor(s) of a layer at a given node.

Arguments:

  • node_index: Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A mask tensor (or list of tensors if the layer has multiple outputs).

get_output_shape_at

get_output_shape_at(
    node_index
)

Retrieves the output shape(s) of a layer at a given node.

Arguments:

  • node_index: Integer, index of the node from which to retrieve the attribute. E.g. node_index=0 will correspond to the first time the layer was called.

Returns:

A shape tuple (or list of shape tuples if the layer has multiple outputs).

Raises:

  • RuntimeError: If called in Eager mode.

get_updates_for

get_updates_for(
    inputs
)

Retrieves updates relevant to a specific set of inputs.

Arguments:

  • inputs: Input tensor or list/tuple of input tensors.

Returns:

List of update ops of the layer that depend on inputs.

get_weights

get_weights()

Returns the current weights of the layer.

Returns:

Weights values as a list of numpy arrays.

load_weights

load_weights(
    filepath, by_name=False, skip_mismatch=False
)

Loads all layer weights, either from a TensorFlow or an HDF5 weight file.

If by_name is False weights are loaded based on the network's topology. This means the architecture should be the same as when the weights were saved. Note that layers that don't have weights are not taken into account in the topological ordering, so adding or removing layers is fine as long as they don't have weights.

If by_name is True, weights are loaded into layers only if they share the same name. This is useful for fine-tuning or transfer-learning models where some of the layers have changed.

Only topological loading (by_name=False) is supported when loading weights from the TensorFlow format. Note that topological loading differs slightly between TensorFlow and HDF5 formats for user-defined classes inheriting from tf.keras.Model: HDF5 loads based on a flattened list of weights, while the TensorFlow format loads based on the object-local names of attributes to which layers are assigned in the Model's constructor.

Arguments:

  • filepath: String, path to the weights file to load. For weight files in TensorFlow format, this is the file prefix (the same as was passed to save_weights).
  • by_name: Boolean, whether to load weights by name or by topological order. Only topological loading is supported for weight files in TensorFlow format.
  • skip_mismatch: Boolean, whether to skip loading of layers where there is a mismatch in the number of weights, or a mismatch in the shape of the weight (only valid when by_name=True).

Returns:

When loading a weight file in TensorFlow format, returns the same status object as tf.train.Checkpoint.restore. When graph building, restore ops are run automatically as soon as the network is built (on first call for user-defined classes inheriting from Model, immediately if it is already built).

When loading weights in HDF5 format, returns None.

Raises:

  • ImportError: If h5py is not available and the weight file is in HDF5 format.
  • ValueError: If skip_mismatch is set to True when by_name is False.

reset_states

reset_states()

save

save(
    filepath, overwrite=True, include_optimizer=True, save_format=None,
    signatures=None, options=None
)

Saves the model to Tensorflow SavedModel or a single HDF5 file.

The savefile includes:

  • The model architecture, allowing to re-instantiate the model.
  • The model weights.
  • The state of the optimizer, allowing to resume training exactly where you left off.

This allows you to save the entirety of the state of a model in a single file.

Saved models can be reinstantiated via keras.models.load_model. The model returned by load_model is a compiled model ready to be used (unless the saved model was never compiled in the first place).

Models built with the Sequential and Functional API can be saved to both the HDF5 and SavedModel formats. Subclassed models can only be saved with the SavedModel format.

Arguments:

  • filepath: String, path to SavedModel or H5 file to save the model.
  • overwrite: Whether to silently overwrite any existing file at the target location, or provide the user with a manual prompt.
  • include_optimizer: If True, save optimizer's state together.
  • save_format: Either 'tf' or 'h5', indicating whether to save the model to Tensorflow SavedModel or HDF5. Defaults to 'tf' in TF 2.X, and 'h5' in TF 1.X.
  • signatures: Signatures to save with the SavedModel. Applicable to the 'tf' format only. Please see the signatures argument in tf.saved_model.save for details.
  • options: Optional tf.saved_model.SaveOptions object that specifies options for saving to SavedModel.

Example:

from keras.models import load_model

model.save('my_model.h5')  # creates a HDF5 file 'my_model.h5'
del model  # deletes the existing model

# returns a compiled model
# identical to the previous one
model = load_model('my_model.h5')

save_weights

save_weights(
    filepath, overwrite=True, save_format=None
)

Saves all layer weights.

Either saves in HDF5 or in TensorFlow format based on the save_format argument.

When saving in HDF5 format, the weight file has: - layer_names (attribute), a list of strings (ordered names of model layers). - For every layer, a group named layer.name - For every such layer group, a group attribute weight_names, a list of strings (ordered names of weights tensor of the layer). - For every weight in the layer, a dataset storing the weight value, named after the weight tensor.

When saving in TensorFlow format, all objects referenced by the network are saved in the same format as tf.train.Checkpoint, including any Layer instances or Optimizer instances assigned to object attributes. For networks constructed from inputs and outputs using tf.keras.Model(inputs, outputs), Layer instances used by the network are tracked/saved automatically. For user-defined classes which inherit from tf.keras.Model, Layer instances must be assigned to object attributes, typically in the constructor. See the documentation of tf.train.Checkpoint and tf.keras.Model for details.

While the formats are the same, do not mix save_weights and tf.train.Checkpoint. Checkpoints saved by Model.save_weights should be loaded using Model.load_weights. Checkpoints saved using tf.train.Checkpoint.save should be restored using the corresponding tf.train.Checkpoint.restore. Prefer tf.train.Checkpoint over save_weights for training checkpoints.

The TensorFlow format matches objects and variables by starting at a root object, self for save_weights, and greedily matching attribute names. For Model.save this is the Model, and for Checkpoint.save this is the Checkpoint even if the Checkpoint has a model attached. This means saving a tf.keras.Model using save_weights and loading into a tf.train.Checkpoint with a Model attached (or vice versa) will not match the Model's variables. See the guide to training checkpoints for details on the TensorFlow format.

Arguments:

  • filepath: String, path to the file to save the weights to. When saving in TensorFlow format, this is the prefix used for checkpoint files (multiple files are generated). Note that the '.h5' suffix causes weights to be saved in HDF5 format.
  • overwrite: Whether to silently overwrite any existing file at the target location, or provide the user with a manual prompt.
  • save_format: Either 'tf' or 'h5'. A filepath ending in '.h5' or '.keras' will default to HDF5 if save_format is None. Otherwise None defaults to 'tf'.

Raises:

  • ImportError: If h5py is not available when attempting to save in HDF5 format.
  • ValueError: For invalid/unknown format arguments.

set_weights

set_weights(
    weights
)

Sets the weights of the layer, from Numpy arrays.

Arguments:

  • weights: a list of Numpy arrays. The number of arrays and their shape must match number of the dimensions of the weights of the layer (i.e. it should match the output of get_weights).

Raises:

  • ValueError: If the provided weights list does not match the layer's specifications.

summary

summary(
    line_length=None, positions=None, print_fn=None
)

Prints a string summary of the network.

Arguments:

  • line_length: Total length of printed lines (e.g. set this to adapt the display to different terminal window sizes).
  • positions: Relative or absolute positions of log elements in each line. If not provided, defaults to [.33, .55, .67, 1.].
  • print_fn: Print function to use. Defaults to print. It will be called on each line of the summary. You can set it to a custom function in order to capture the string summary.

Raises:

  • ValueError: if summary() is called before the model is built.

to_json

to_json(
    **kwargs
)

Returns a JSON string containing the network configuration.

To load a network from a JSON save file, use keras.models.model_from_json(json_string, custom_objects={}).

Arguments:

  • **kwargs: Additional keyword arguments to be passed to json.dumps().

Returns:

A JSON string.

to_yaml

to_yaml(
    **kwargs
)

Returns a yaml string containing the network configuration.

To load a network from a yaml save file, use keras.models.model_from_yaml(yaml_string, custom_objects={}).

custom_objects should be a dictionary mapping the names of custom losses / layers / etc to the corresponding functions / classes.

Arguments:

  • **kwargs: Additional keyword arguments to be passed to yaml.dump().

Returns:

A YAML string.

Raises:

  • ImportError: if yaml module is not found.

with_name_scope

@classmethod
with_name_scope(
    cls, method
)

Decorator to automatically enter the module name scope.

class MyModule(tf.Module):
  @tf.Module.with_name_scope
  def __call__(self, x):
    if not hasattr(self, 'w'):
      self.w = tf.Variable(tf.random.normal([x.shape[1], 64]))
    return tf.matmul(x, self.w)

Using the above module would produce tf.Variables and tf.Tensors whose names included the module name:

mod = MyModule()
mod(tf.ones([8, 32]))
# ==> <tf.Tensor: ...>
mod.w
# ==> <tf.Variable ...'my_module/w:0'>

Args:

  • method: The method to wrap.

Returns:

The original method wrapped such that it enters the module's name scope.