Have a question? Connect with the community at the TensorFlow Forum Visit Forum

tfnlp.layers.MobileBertMaskedLM

Masked language model network head for BERT modeling.

This layer implements a masked language model based on the provided transformer based encoder. It assumes that the encoder network being passed has a "get_embedding_table()" method. Different from canonical BERT's masked LM layer, when the embedding width is smaller than hidden_size, it adds an extra output weights in shape [vocab_size, (hidden_size - embedding_width)].

embedding_table The embedding table from encoder network.
activation The activation, if any, for the dense layer.
initializer The initializer for the dense layer. Defaults to a Glorot uniform initializer.
output The output style for this layer. Can be either 'logits' or 'predictions'.
**kwargs keyword arguments.

Methods

call

View source

This is where the layer's logic lives.

Note here that call() method in tf.keras is little bit different from keras API. In keras API, you can pass support masking for layers as additional arguments. Whereas tf.keras has compute_mask() method to support masking.

Args
inputs Input tensor, or list/tuple of input tensors.
*args Additional positional arguments. Currently unused.
**kwargs Additional keyword arguments. Currently unused.

Returns
A tensor or list/tuple of tensors.