Layer normalization layer (Ba et al., 2016).

Inherits From: Layer, Module

Used in the notebooks

Used in the tutorials

Normalize the activations of the previous layer for each given example in a batch independently, rather than across a batch like Batch Normalization. i.e. applies a transformation that maintains the mean activation within each example close to 0 and the activation standard deviation close to 1.

Given a tensor inputs, moments are calculated and normalization is performed across the axes specified in axis.


data = tf.constant(np.arange(10).reshape(5, 2) * 10, dtype=tf.float32)