Registration is open for TensorFlow Dev Summit 2020 Learn more

tf.keras.experimental.PeepholeLSTMCell

TensorFlow 1 version View source on GitHub

Class PeepholeLSTMCell

Equivalent to LSTMCell class but adds peephole connections.

Inherits From: LSTMCell

Peephole connections allow the gates to utilize the previous internal state as well as the previous hidden state (which is what LSTMCell is limited to). This allows PeepholeLSTMCell to better learn precise timings over LSTMCell.

From Gers et al.:

"We find that LSTM augmented by 'peephole connections' from its internal cells to its multiplicative gates can learn the fine distinction between sequences of spikes spaced either 50 or 49 time steps apart without the help of any short training exemplars."

The peephole implementation is based on:

Long short-term memory recurrent neural network architectures for large scale acoustic modeling.

Example:

# Create 2 PeepholeLSTMCells
peephole_lstm_cells = [PeepholeLSTMCell(size) for size in [128, 256]]
# Create a layer composed sequentially of the peephole LSTM cells.
layer = RNN(peephole_lstm_cells)
input = keras.Input((timesteps, input_dim))
output = layer(input)