TF 2.0 is out! Get hands-on practice at TF World, Oct 28-31. Use code TF20 for 20% off select passes. Register now

Module: tfa.seq2seq.attention_wrapper

View source on GitHub

A powerful dynamic attention wrapper object.


class AttentionMechanism

class AttentionWrapper: Wraps another RNNCell with attention.

class AttentionWrapperState: namedtuple storing the state of a AttentionWrapper.

class BahdanauAttention: Implements Bahdanau-style (additive) attention.

class BahdanauMonotonicAttention: Monotonic attention mechanism with Bahadanau-style energy function.

class LuongAttention: Implements Luong-style (multiplicative) attention scoring.

class LuongMonotonicAttention: Monotonic attention mechanism with Luong-style energy function.


hardmax(...): Returns batched one-hot vectors.

monotonic_attention(...): Compute monotonic attention distribution from choosing probabilities.

safe_cumprod(...): Computes cumprod of x in logspace using cumsum to avoid underflow.