The main differences between a TF-Agents Network and a Keras Layer include:
networks keep track of their underlying layers, explicitly represent RNN-like
state in inputs and outputs, and simplify variable creation and clone
When calling a network net, typically one passes data through it via:
outputs, next_state = net(observation, network_state=...)
outputs, next_state = net(observation, step_type=..., network_state=...)
outputs, next_state = net(observation) # net.call must fill an empty state
outputs, next_state = net(observation, step_type=...)
outputs, next_state = net(
observation, step_type=..., network_state=..., learning=...)
To force construction of a network's variables:
net.create_variables(input_tensor_spec=...) # To provide an input spec
net.create_variables(training=True) # Provide extra kwargs
To create a copy of the network:
cloned_net = net.copy()
cloned_net.variables # Raises ValueError: cloned net does not share weights.
cloned_net.variables # Now new variables have been created.
A nest of tf.TypeSpec representing the
input observations. Optional. If not provided, create_variables()
will fail unless a spec is provided.
A nest of tensor_spec.TensorSpec representing the state
needed by the network. Default is (), which means no state.
(Optional.) A string representing the name of the network.
Returns the spec of the input to the network of type InputSpec.
Get the list of all (nested) sub-layers used in this Network.