Join us at TensorFlow World, Oct 28-31. Use code TF20 for 20% off select passes. Register now

nsl.lib.normalize

View source on GitHub

Normalizes the values in tensor with respect to the specified vector norm.

nsl.lib.normalize(
    tensor,
    norm_type,
    epsilon=1e-06
)

This op takes into account the first axis being the batch dimension, and calculates the norm over all other axises. For example: assuming 'tensor' = tf.constant(1.0, shape=[2, 3, 4]), its L2 norm (calculated along all the dims other than the first dim) will be [[sqrt(12)], [sqrt(12)]]; therefore, this tensor will be normalized by dividing [[sqrt(12)], [sqrt(12)]].

Note that tf.norm is not used here since it only allows the norm to be calculated over one axis, not multiple axes.

Args:

  • tensor: a tensor to be normalized. Can have any shape with the first axis being the batch dimension that will not be normalized across.
  • norm_type: one of configs.NormType, the type of vector norm.
  • epsilon: a lower bound value for the norm to avoid division by 0.

Returns:

A normalized tensor with the same shape and type as 'tensor'.