# tf.compat.v1.keras.layers.BatchNormalization

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/keras-team/keras/tree/v2.13.1/keras/layers/normalization/batch_normalization_v1.py#L24-L31) |

Layer that normalizes its inputs.

Inherits From: [`Layer`](https://www.tensorflow.org/versions/r2.13/api_docs/python/tf/keras/layers/Layer), [`Module`](https://www.tensorflow.org/versions/r2.13/api_docs/python/tf/Module)

    tf.compat.v1.keras.layers.BatchNormalization(
        *args, **kwargs
    )

Batch normalization applies a transformation that maintains the mean output
close to 0 and the output standard deviation close to 1.

Importantly, batch normalization works differently during training and
during inference.

**During training** (i.e. when using `fit()` or when calling the layer/model
with the argument `training=True`), the layer normalizes its output using
the mean and standard deviation of the current batch of inputs. That is to
say, for each channel being normalized, the layer returns
`gamma * (batch - mean(batch)) / sqrt(var(batch) + epsilon) + beta`, where:

- `epsilon` is small constant (configurable as part of the constructor arguments)
- `gamma` is a learned scaling factor (initialized as 1), which can be disabled by passing `scale=False` to the constructor.
- `beta` is a learned offset factor (initialized as 0), which can be disabled by passing `center=False` to the constructor.

**During inference** (i.e. when using `evaluate()` or `predict()`) or when
calling the layer/model with the argument `training=False` (which is the
default), the layer normalizes its output using a moving average of the
mean and standard deviation of the batches it has seen during training. That
is to say, it returns
`gamma * (batch - self.moving_mean) / sqrt(self.moving_var+epsilon) + beta`.

`self.moving_mean` and `self.moving_var` are non-trainable variables that
are updated each time the layer in called in training mode, as such:

- `moving_mean = moving_mean * momentum + mean(batch) * (1 - momentum)`
- `moving_var = moving_var * momentum + var(batch) * (1 - momentum)`

As such, the layer will only normalize its inputs during inference
*after having been trained on data that has similar statistics as the
inference data*.

<br />

<br />

<br />

| ## Args ||
|---|---|
| `axis` | Integer or a list of integers, the axis that should be normalized (typically the features axis). For instance, after a `Conv2D` layer with `data_format="channels_first"`, set `axis=1` in `BatchNormalization`. |
| `momentum` | Momentum for the moving average. |
| `epsilon` | Small float added to variance to avoid dividing by zero. |
| `center` | If True, add offset of `beta` to normalized tensor. If False, `beta` is ignored. |
| `scale` | If True, multiply by `gamma`. If False, `gamma` is not used. When the next layer is linear (also e.g. [`nn.relu`](https://www.tensorflow.org/api_docs/python/tf/nn/relu)), this can be disabled since the scaling will be done by the next layer. |
| `beta_initializer` | Initializer for the beta weight. |
| `gamma_initializer` | Initializer for the gamma weight. |
| `moving_mean_initializer` | Initializer for the moving mean. |
| `moving_variance_initializer` | Initializer for the moving variance. |
| `beta_regularizer` | Optional regularizer for the beta weight. |
| `gamma_regularizer` | Optional regularizer for the gamma weight. |
| `beta_constraint` | Optional constraint for the beta weight. |
| `gamma_constraint` | Optional constraint for the gamma weight. |
| `renorm` | Whether to use [Batch Renormalization](https://arxiv.org/abs/1702.03275). This adds extra variables during training. The inference is the same for either value of this parameter. |
| `renorm_clipping` | A dictionary that may map keys 'rmax', 'rmin', 'dmax' to scalar `Tensors` used to clip the renorm correction. The correction `(r, d)` is used as `corrected_value = normalized_value * r + d`, with `r` clipped to \[rmin, rmax\], and `d` to \[-dmax, dmax\]. Missing rmax, rmin, dmax are set to inf, 0, inf, respectively. |
| `renorm_momentum` | Momentum used to update the moving means and standard deviations with renorm. Unlike `momentum`, this affects training and should be neither too small (which would add noise) nor too large (which would give stale estimates). Note that `momentum` is still applied to get the means and variances for inference. |
| `fused` | if `True`, use a faster, fused implementation, or raise a ValueError if the fused implementation cannot be used. If `None`, use the faster implementation if possible. If False, do not used the fused implementation. Note that in TensorFlow 1.x, the meaning of `fused=True` is different: if `False`, the layer uses the system-recommended implementation. You cannot use `fused=True` if a mask is passed in the `call()` method. |
| `trainable` | Boolean, if `True` the variables will be marked as trainable. |
| `virtual_batch_size` | An `int`. By default, `virtual_batch_size` is `None`, which means batch normalization is performed across the whole batch. When `virtual_batch_size` is not `None`, instead perform "Ghost Batch Normalization", which creates virtual sub-batches which are each normalized separately (with shared gamma, beta, and moving statistics). Must divide the actual batch size during execution. |
| `adjustment` | A function taking the `Tensor` containing the (dynamic) shape of the input tensor and returning a pair (scale, bias) to apply to the normalized values (before gamma and beta), only during training. For example, if `axis=-1`, `adjustment = lambda shape: ( tf.random.uniform(shape[-1:], 0.93, 1.07), tf.random.uniform(shape[-1:], -0.1, 0.1))` will scale the normalized value by up to 7% up or down, then shift the result by up to 0.1 (with independent scaling and bias for each feature but shared across all examples), and finally apply gamma and/or beta. If `None`, no adjustment is applied. Cannot be specified if virtual_batch_size is specified. |
| `synchronized` | If True, synchronizes the global batch statistics (mean and variance) for the layer across all devices at each training step in a distributed training strategy. If False, each replica uses its own local batch statistics. Only relevant when used inside a [`tf.distribute`](https://www.tensorflow.org/versions/r2.13/api_docs/python/tf/distribute) strategy. |

<br />

<br />

<br />

<br />

| ## Call arguments ||
|---|---|
| `inputs` | Input tensor (of any rank). |
| `training` | Python boolean indicating whether the layer should behave in training mode or in inference mode. <br /> - `training=True`: The layer will normalize its inputs using the mean and variance of the current batch of inputs. - `training=False`: The layer will normalize its inputs using the mean and variance of its moving statistics, learned during training. |
| `mask` | Binary tensor of shape broadcastable to `inputs` tensor, indicating the positions for which the mean and variance should be computed. |

Input shape: Arbitrary. Use the keyword argument `input_shape` (tuple of
integers, does not include the samples axis) when using this layer as the
first layer in a model.

Output shape: Same shape as input.

<br />

<br />

| ## Reference ||
|---|---|
| <br /> - [Ioffe and Szegedy, 2015](https://arxiv.org/abs/1502.03167). ||

<br />