# tf.contrib.legacy_seq2seq.attention_decoder

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py#L537-L704) |

RNN decoder with attention for the sequence-to-sequence model.

    tf.contrib.legacy_seq2seq.attention_decoder(
        decoder_inputs, initial_state, attention_states, cell, output_size=None,
        num_heads=1, loop_function=None, dtype=None, scope=None,
        initial_state_attention=False
    )

In this context "attention" means that, during decoding, the RNN can look up
information in the additional tensor attention_states, and it does this by
focusing on a few entries from the tensor. This model has proven to yield
especially good results in a number of sequence-to-sequence tasks. This
implementation is based on <http://arxiv.org/abs/1412.7449> (see below for
details). It is recommended for complex sequence-to-sequence tasks.

<br />

<br />

<br />

| ## Args ||
|---|---|
| `decoder_inputs` | A list of 2D Tensors \[batch_size x input_size\]. |
| `initial_state` | 2D Tensor \[batch_size x cell.state_size\]. |
| `attention_states` | 3D Tensor \[batch_size x attn_length x attn_size\]. |
| `cell` | tf.compat.v1.nn.rnn_cell.RNNCell defining the cell function and size. |
| `output_size` | Size of the output vectors; if None, we use cell.output_size. |
| `num_heads` | Number of attention heads that read from attention_states. |
| `loop_function` | If not None, this function will be applied to i-th output in order to generate i+1-th input, and decoder_inputs will be ignored, except for the first element ("GO" symbol). This can be used for decoding, but also for training to emulate <http://arxiv.org/abs/1506.03099> Signature -- loop_function(prev, i) = next \* prev is a 2D Tensor of shape \[batch_size x output_size\], \* i is an integer, the step number (when advanced control is needed), \* next is a 2D Tensor of shape \[batch_size x input_size\]. |
| `dtype` | The dtype to use for the RNN initial state (default: tf.float32). |
| `scope` | VariableScope for the created subgraph; default: "attention_decoder". |
| `initial_state_attention` | If False (default), initial attentions are zero. If True, initialize the attentions from the initial state and attention states -- useful when we wish to resume decoding from a previously stored decoder state and attention states. |

<br />

<br />

<br />

<br />

| ## Returns ||
|---|---|
| A tuple of the form (outputs, state), where: outputs: A list of the same length as decoder_inputs of 2D Tensors of shape \[batch_size x output_size\]. These represent the generated outputs. Output i is computed from input i (which is either the i-th element of decoder_inputs or loop_function(output {i-1}, i)) as follows. First, we run the cell on a combination of the input and previous attention masks: cell_output, new_state = cell(linear(input, prev_attn), prev_state). Then, we calculate new attention masks: new_attn = softmax(V\^T \* tanh(W \* attention_states + U \* new_state)) and then we calculate the output: output = linear(cell_output, new_attn). state: The state of each decoder cell the final time-step. It is a 2D Tensor of shape \[batch_size x cell.state_size\]. ||

<br />

<br />

<br />

<br />

| ## Raises ||
|---|---|
| `ValueError` | when num_heads is not positive, there are no inputs, shapes of attention_states are not set, or input size cannot be inferred from the input. |

<br />