Missed TensorFlow Dev Summit? Check out the video playlist. Watch recordings

tf.compat.v1.nn.fused_batch_norm

View source on GitHub

Batch normalization.

tf.compat.v1.nn.fused_batch_norm(
    x, scale, offset, mean=None, variance=None, epsilon=0.001, data_format='NHWC',
    is_training=True, name=None, exponential_avg_factor=1.0
)

See Source: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift; S. Ioffe, C. Szegedy.

Args:

  • x: Input Tensor of 4 dimensions.
  • scale: A Tensor of 1 dimension for scaling.
  • offset: A Tensor of 1 dimension for bias.
  • mean: A Tensor of 1 dimension for population mean. The shape and meaning of this argument depends on the value of is_training and exponential_avg_factor as follows: is_trainingFalse (inference): Mean must be a Tensor of the same shape as scale containing the estimated population mean computed during training. is_trainingTrue and exponential_avg_factor == 1.0: Mean must be None. is_training==True and exponential_avg_factor != 1.0: Mean must be a Tensor of the same shape as scale containing the exponential running mean.
  • variance: A Tensor of 1 dimension for population variance. The shape and meaning of this argument depends on the value of is_training and exponential_avg_factor as follows: is_trainingFalse (inference): Variance must be a Tensor of the same shape as scale containing the estimated population variance computed during training. is_trainingTrue and exponential_avg_factor == 1.0: Variance must be None. is_training==True and exponential_avg_factor != 1.0: Variance must be a Tensor of the same shape as scale containing the exponential running variance.
  • epsilon: A small float number added to the variance of x.
  • data_format: The data format for x. Either "NHWC" (default) or "NCHW".
  • is_training: A bool value to specify if the operation is used for training or inference.
  • name: A name for this operation (optional).
  • exponential_avg_factor: A float number (usually between 0 and 1) used for controlling the decay of the running population average of mean and variance. If set to 1.0, the current batch average is returned.

Returns:

  • y: A 4D Tensor for the normalized, scaled, offsetted x.
  • running_mean: A 1D Tensor for the exponential running mean of x. The output value is (1 - exponential_avg_factor) * mean + exponential_avg_factor * batch_mean), where batch_mean is the mean of the current batch in x.
  • running_var: A 1D Tensor for the exponential running variance The output value is (1 - exponential_avg_factor) * variance + exponential_avg_factor * batch_variance), where batch_variance is the variance of the current batch in x.

References:

Batch Normalization - Accelerating Deep Network Training by Reducing Internal Covariate Shift: Ioffe et al., 2015 (pdf)