{ }
View source on GitHub |
Dot-product attention layer, a.k.a. Luong-style attention.
Inherits From: Layer
, Operation
tf.keras.layers.Attention(
use_scale=False,
score_mode='dot',
dropout=0.0,
seed=None,
**kwargs
)
Inputs are a list with 2 or 3 elements:
- A
query
tensor of shape(batch_size, Tq, dim)
. - A
value
tensor of shape(batch_size, Tv, dim)
. - A optional
key
tensor of shape(batch_size, Tv, dim)
. If none supplied,value
will be used as akey
.
The calculation follows the steps:
- Calculate attention scores using
query
andkey
with shape(batch_size, Tq, Tv)
. - Use scores to calculate a softmax distribution with shape
(batch_size, Tq, Tv)
. - Use the softmax distribution to create a linear combination of
value
with shape(batch_size, Tq, dim)
.
Output | |
---|---|
Attention outputs of shape (batch_size, Tq, dim) .
(Optional) Attention scores after masking and softmax with shape
(batch_size, Tq, Tv) .
|
Methods
from_config
@classmethod
from_config( config )
Creates a layer from its config.
This method is the reverse of get_config
,
capable of instantiating the same layer from the config
dictionary. It does not handle layer connectivity
(handled by Network), nor weights (handled by set_weights
).
Args | |
---|---|
config
|
A Python dictionary, typically the output of get_config. |
Returns | |
---|---|
A layer instance. |
symbolic_call
symbolic_call(
*args, **kwargs
)