<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/models/blob/v2.20.0/official/nlp/modeling/models/t5.py#L1005-L1034) |

Transformer parameters.

    tfm.nlp.models.T5TransformerParams(
        num_layers: int,
        d_model: int,
        d_kv: int,
        num_heads: int,
        d_ff: int,
        vocab_size: int,
        target_vocab_size: Optional[int] = None,
        dropout_rate: float = 0.0,
        layer_norm_epsilon: float = 1e-06,
        shared_embedding: bool = False,
        vocab_embeddings_initializer: Optional[Initializer] = None,
        relative_attention_num_buckets: int = 32,
        relative_attention_max_distance: int = 128,
        relative_embeddings_initializer: Optional[Initializer] = None,
        weight_initializer: Optional[Initializer] = https://www.tensorflow.org/api_docs/python/tfm/nlp/models/T5TransformerParams#weight_initializer,
        bias_initializer: Optional[Initializer] = None,
        rescale_query: bool = False,
        bidirectional: bool = True,
        ffn_activations: Sequence[str] = https://www.tensorflow.org/api_docs/python/tfm/nlp/models/T5TransformerParams#ffn_activations,
        logits_via_embedding: bool = True,
        num_decoder_layers: Optional[int] = None,
        one_hot_embedding: bool = True,
        layer_sharing: bool = False,
        use_shared_relative_position_bias: bool = True,
        return_attention_scores: bool = False
    )

<br />

<br />

<br />

| ## Attributes ||
|---|---|
| `num_layers` | Dataclass field |
| `d_model` | Dataclass field |
| `d_kv` | Dataclass field |
| `num_heads` | Dataclass field |
| `d_ff` | Dataclass field |
| `vocab_size` | Dataclass field |
| `target_vocab_size` | Dataclass field |
| `dropout_rate` | Dataclass field |
| `layer_norm_epsilon` | Dataclass field |
| `shared_embedding` | Dataclass field |
| `vocab_embeddings_initializer` | Dataclass field |
| `relative_attention_num_buckets` | Dataclass field |
| `relative_attention_max_distance` | Dataclass field |
| `relative_embeddings_initializer` | Dataclass field |
| `weight_initializer` | Dataclass field |
| `bias_initializer` | Dataclass field |
| `rescale_query` | Dataclass field |
| `bidirectional` | Dataclass field |
| `ffn_activations` | Dataclass field |
| `logits_via_embedding` | Dataclass field |
| `num_decoder_layers` | Dataclass field |
| `one_hot_embedding` | Dataclass field |
| `layer_sharing` | Dataclass field |
| `use_shared_relative_position_bias` | Dataclass field |
| `return_attention_scores` | Dataclass field |

<br />

## Methods

### `weight_initializer`

    weight_initializer(
        dtype=None, **kwargs
    )

He normal initializer.

Also available via the shortcut function
[`tf.keras.initializers.he_normal`](https://www.tensorflow.org/api_docs/python/tf/keras/initializers/HeNormal).

It draws samples from a truncated normal distribution centered on 0 with
`stddev = sqrt(2 / fan_in)` where `fan_in` is the number of input units in
the weight tensor.

#### Examples:

    # Standalone usage:
    initializer = tf.keras.initializers.HeNormal()
    values = initializer(shape=(2, 2))

    # Usage in a TF-Keras layer:
    initializer = tf.keras.initializers.HeNormal()
    layer = tf.keras.layers.Dense(3, kernel_initializer=initializer)

<br />

<br />

<br />

| Args ||
|---|---|
| `seed` | A Python integer. Used to make the behavior of the initializer deterministic. Note that a seeded initializer will not produce the same random values across multiple calls, but multiple initializers will produce the same sequence when constructed with the same seed value. |

<br />

<br />

<br />

| References ||
|---|---|
| <br /> - [He et al., 2015](https://arxiv.org/abs/1502.01852) ||

<br />

### `__eq__`

    __eq__(
        other
    )

Return self==value.

<br />

<br />

<br />

| ## Class Variables ||
|---|---|
| bias_initializer | `None` |
| bidirectional | `True` |
| dropout_rate | `0.0` |
| ffn_activations | `('relu',)` |
| layer_norm_epsilon | `1e-06` |
| layer_sharing | `False` |
| logits_via_embedding | `True` |
| num_decoder_layers | `None` |
| one_hot_embedding | `True` |
| relative_attention_max_distance | `128` |
| relative_attention_num_buckets | `32` |
| relative_embeddings_initializer | `None` |
| rescale_query | `False` |
| return_attention_scores | `False` |
| shared_embedding | `False` |
| target_vocab_size | `None` |
| use_shared_relative_position_bias | `True` |
| vocab_embeddings_initializer | `None` |

<br />