nsl.lib.normalize

View source on GitHub

Normalizes the values in tensor with respect to a specified vector norm.

This op assumes that the first axis of tensor is the batch dimension, and calculates the norm over all other axes. For example, if tensor is tf.constant(1.0, shape=[2, 3, 4]), its L2 norm (calculated along all the dimensions other than the first dimension) will be [[sqrt(12)], [sqrt(12)]]. Hence, this tensor will be normalized by dividing by [[sqrt(12)], [sqrt(12)]].

Note that tf.norm is not used here since it only allows the norm to be calculated over one axis, not multiple axes.

tensor a tensor to be normalized. Can have any shape with the first axis being the batch dimension that will not be normalized across.
norm_type one of nsl.configs.NormType, the type of vector norm.
epsilon a lower bound value for the norm to avoid division by 0.

A normalized tensor with the same shape and type as tensor.