tfm.nlp.encoders.BigBirdEncoderConfig

BigBird encoder configuration.

Inherits From: Config, ParamsDict

BUILDER

default_params Dataclass field
restrictions Dataclass field
vocab_size Dataclass field
hidden_size Dataclass field
num_layers Dataclass field
num_attention_heads Dataclass field
hidden_activation Dataclass field
intermediate_size Dataclass field
dropout_rate Dataclass field
attention_dropout_rate Dataclass field
norm_first Dataclass field
max_position_embeddings Dataclass field
num_rand_blocks Dataclass field
block_size Dataclass field
type_vocab_size Dataclass field
initializer_range Dataclass field
embedding_width Dataclass field
use_gradient_checkpointing Dataclass field

Methods

as_dict

View source

Returns a dict representation of params_dict.ParamsDict.

For the nested params_dict.ParamsDict, a nested dict will be returned.

from_args

View source

Builds a config from the given list of arguments.

from_json

View source

Wrapper for from_yaml.

from_yaml

View source

get

View source

Accesses through built-in dictionary get method.

lock

View source

Makes the ParamsDict immutable.

override

View source

Override the ParamsDict with a set of given params.

Args
override_params a dict or a ParamsDict specifying the parameters to be overridden.
is_strict a boolean specifying whether override is strict or not. If True, keys in override_params must be present in the ParamsDict. If False, keys in override_params can be different from what is currently defined in the ParamsDict. In this case, the ParamsDict will be extended to include the new keys.

replace

View source

Overrides/returns a unlocked copy with the current config unchanged.

validate

View source

Validate the parameters consistency based on the restrictions.

This method validates the internal consistency using the pre-defined list of restrictions. A restriction is defined as a string which specfiies a binary operation. The supported binary operations are {'==', '!=', '<', '<=', '>', '>='}. Note that the meaning of these operators are consistent with the underlying Python immplementation. Users should make sure the define restrictions on their type make sense.

For example, for a ParamsDict like the following

a:
  a1: 1
  a2: 2
b:
  bb:
    bb1: 10
    bb2: 20
  ccc:
    a1: 1
    a3: 3

one can define two restrictions like this ['a.a1 == b.ccc.a1', 'a.a2 <= b.bb.bb2']

What it enforces are:

  • a.a1 = 1 == b.ccc.a1 = 1
  • a.a2 = 2 <= b.bb.bb2 = 20

Raises
KeyError if any of the following happens (1) any of parameters in any of restrictions is not defined in ParamsDict, (2) any inconsistency violating the restriction is found.
ValueError if the restriction defined in the string is not supported.

__contains__

View source

Implements the membership test operator.

__eq__

IMMUTABLE_TYPES (<class 'str'>, <class 'int'>, <class 'float'>, <class 'bool'>, <class 'NoneType'>)
RESERVED_ATTR ['_locked', '_restrictions']
SEQUENCE_TYPES (<class 'list'>, <class 'tuple'>)
attention_dropout_rate 0.1
block_size 64
default_params None
dropout_rate 0.1
embedding_width None
hidden_activation 'gelu'
hidden_size 768
initializer_range 0.02
intermediate_size 3072
max_position_embeddings 4096
norm_first False
num_attention_heads 12
num_layers 12
num_rand_blocks 3
restrictions None
type_vocab_size 16
use_gradient_checkpointing False
vocab_size 50358