<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/models/blob/v2.20.0/official/nlp/modeling/models/xlnet.py#L79-L152) |

XLNet-based pretrainer.

    tfm.nlp.models.XLNetPretrainer(
        network: Union[tf_keras.layers.Layer, tf_keras.Model],
        mlm_activation=None,
        mlm_initializer='glorot_uniform',
        name: Optional[str] = None,
        **kwargs
    )

This is an implementation of the network structure surrounding a
Transformer-XL encoder as described in "XLNet: Generalized Autoregressive
Pretraining for Language Understanding" (<https://arxiv.org/abs/1906.08237>).

<br />

<br />

<br />

| ## Args ||
|---|---|
| `network` | An XLNet/Transformer-XL based network. This network should output a sequence output and list of `state` tensors. |
| `mlm_activation` | The activation (if any) to use in the Masked LM network. If None, then no activation will be used. |
| `mlm_initializer` | The initializer (if any) to use in the masked LM. Defaults to a Glorot uniform initializer. |

<br />

<br />

<br />

<br />

| ## Attributes ||
|---|---|
| `checkpoint_items` | <br /> <br /> |

<br />

## Methods

### `call`

[View source](https://github.com/tensorflow/models/blob/v2.20.0/official/nlp/modeling/models/xlnet.py#L120-L141)

    call(
        inputs: Mapping[str, Any]
    )

Calls the model on new inputs and returns the outputs as tensors.

In this case `call()` just reapplies
all ops in the graph to the new inputs
(e.g. build a new computational graph from the provided inputs).

> [!NOTE]
> **Note:** This method should not be called directly. It is only meant to be overridden when subclassing [`tf.keras.Model`](https://www.tensorflow.org/api_docs/python/tf/keras/Model). To call a model on an input, always use the `__call__()` method, i.e. `model(inputs)`, which relies on the underlying `call()` method.

<br />

<br />

<br />

| Args ||
|---|---|
| `inputs` | Input tensor, or dict/list/tuple of input tensors. |
| `training` | Boolean or boolean scalar tensor, indicating whether to run the `Network` in training mode or inference mode. |
| `mask` | A mask or list of masks. A mask can be either a boolean tensor or None (no mask). For more details, check the guide [here](https://www.tensorflow.org/guide/keras/masking_and_padding). |

<br />

<br />

<br />

<br />

| Returns ||
|---|---|
| A tensor if there is a single output, or a list of tensors if there are more than one outputs. ||

<br />