The main differences between a TF-Agents Network and a Keras Layer include:
networks keep track of their underlying layers, explicitly represent RNN-like
state in inputs and outputs, and simplify variable creation and clone
When calling a network net, typically one passes data through it via:
outputs, next_state = net(observation, network_state=...)
outputs, next_state = net(observation, step_type=..., network_state=...)
outputs, next_state = net(observation) # net.call must fill an empty state
outputs, next_state = net(observation, step_type=...)
outputs, next_state = net(
observation, step_type=..., network_state=..., learning=...)
To force construction of a network's variables:
net.create_variables(input_tensor_spec=...) # To provide an input spec
net.create_variables(training=True) # Provide extra kwargs
To create a copy of the network:
cloned_net = net.copy()
cloned_net.variables # Raises ValueError: cloned net does not share weights.
cloned_net.variables # Now new variables have been created.
A nest of tf.TypeSpec representing the input
observations. Optional. If not provided, create_variables() will
fail unless a spec is provided.
A nest of tensor_spec.TensorSpec representing the state
needed by the network. Default is (), which means no state.
(Optional.) A string representing the name of the network.
Returns the spec of the input to the network of type InputSpec.
Get the list of all (nested) sub-layers used in this Network.
(Optional). Override or provide an input tensor spec
when creating variables.
Other arguments to network.call(), e.g. training=True.
Output specs - a nested spec calculated from the outputs (excluding any
batch dimensions). If any of the output elements is a tfp Distribution,
the associated spec entry returned is a DistributionSpec.
If no input_tensor_spec is provided, and the network did
not provide one during construction.