tf.keras.layers.GRU

Gated Recurrent Unit - Cho et al. 2014.

Inherits From: GRU, RNN, Layer, Module

Used in the notebooks

Used in the guide Used in the tutorials

See the Keras RNN API guide for details about the usage of RNN API.

Based on available runtime hardware and constraints, this layer will choose different implementations (cuDNN-based or pure-TensorFlow) to maximize the performance. If a GPU is available and all the arguments to the layer meet the requirement of the CuDNN kernel (see below for details), the layer will use a fast cuDNN implementation.

The requirements to use the cuDNN implementation are:

  1. activation == tanh
  2. recurrent_activation == sigmoid
  3. recurrent_dropout == 0
  4. unroll is False
  5. use_bias is True
  6. reset_after is True
  7. Inputs, if use masking, are strictly right-padded.
  8. Eager execution is enabled in the outermost context.

There are two variants of the GRU implementation. The default one is based on v3 and has reset gate applied to hidden state before matrix multiplication. The other one is based on original and has the order reversed.

The second variant is compatible with CuDNNGRU (GPU-only) and allows inference on CPU. Thus it has separate biases for kernel and recurrent_kernel. To use this variant, set 'reset_after'=True and recurrent_activation='sigmoid'.

For example:

inputs = tf.random.normal([32, 10, 8])
gru = tf.keras.layers.GRU(4)
output = gru(inputs)
print(output.shape)
(32, 4)
gru = tf.keras.layers.GRU(4, return_sequences=True, return_state=True)
whole_sequence_output, final_state = gru(inputs)
print(whole_sequence_output.shape)
(32, 10, 4)
print(final_state.shape)
(32, 4)

units Positive integer, dimensionality of the output space.
activation Activation function to use. Default: hyperbolic tangent (tanh). If you pass None, no activation is applied (ie. "linear" activation: a(x) = x).
recurrent_activation Activation function to use for the recurrent step. Default: sigmoid (sigmoid). If you pass None, no activation is applied (ie. "linear" activation: a(x) = x).
use_bias Boolean, (default True), whether the layer uses a bias vector.
kernel_initializer Initializer for the kernel weights matrix, used for the linear transformation of the inputs. Default: glorot_uniform.
recurrent_initializer Initializer for the recurrent_kernel weights matrix, used for the linear transformation of the recurrent state. Default: orthogonal.
bias_initializer Initializer for the bias vector. Default: zeros.
kernel_regularizer Regularizer function applied to the kernel weights matrix. Default: None.
recurrent_regularizer Regularizer function applied to the recurrent_kernel weights matrix. Default: None.
bias_regularizer Regularizer function applied to the bias vector. Default: None.
activity_regularizer Regularizer function applied to the output of the layer (its "activation"). Default: None.
kernel_constraint Constraint function applied to the kernel weights matrix. Default: None.
recurrent_constraint Constraint function applied to the recurrent_kernel weights matrix. Default: None.
bias_constraint Constraint function applied to the bias vector. Default: None.
dropout Float between 0 and 1. Fraction of the units to drop for the linear transformation of the inputs. Default: 0.
recurrent_dropout Float between 0 and 1. Fraction of the units to drop for the linear transformation of the recurrent state. Default: 0.
return_sequences Boolean. Whether to return the last output in the output sequence, or the full sequence. Default: False.
return_state