|TensorFlow 1 version||View source on GitHub|
Layer that normalizes its inputs.
tf.keras.layers.BatchNormalization( axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True, beta_initializer='zeros', gamma_initializer='ones', moving_mean_initializer='zeros', moving_variance_initializer='ones', beta_regularizer=None, gamma_regularizer=None, beta_constraint=None, gamma_constraint=None, **kwargs )
Used in the notebooks
|Used in the guide||Used in the tutorials|
Batch normalization applies a transformation that maintains the mean output close to 0 and the output standard deviation close to 1.
Importantly, batch normalization works differently during training and during inference.
During training (i.e. when using
fit() or when calling the layer/model
with the argument
training=True), the layer normalizes its output using
the mean and standard deviation of the current batch of inputs. That is to
say, for each channel being normalized, the layer returns
gamma * (batch - mean(batch)) / sqrt(var(batch) + epsilon) + beta, where:
epsilonis small constant (configurable as part of the constructor arguments)
gammais a learned scaling factor (initialized as 1), which can be disabled by passing
scale=Falseto the constructor.
betais a learned offset factor (initialized as 0), which can be disabled by passing
center=Falseto the constructor.
During inference (i.e. when using
predict() or when
calling the layer/model with the argument
training=False (which is the
default), the layer normalizes its output using a moving average of the
mean and standard deviation of the batches it has seen during training. That
is to say, it returns
gamma * (batch - self.moving_mean) / sqrt(self.moving_var + epsilon) + beta.
self.moving_var are non-trainable variables that
are updated each time the layer in called in training mode, as such:
moving_mean = moving_mean * momentum + mean(batch) * (1 - momentum)
moving_var = moving_var * momentum + var(batch) * (1 - momentum)
As such, the layer will only normalize its inputs during inference after having been trained on data that has similar statistics as the inference data.
Integer, the axis that should be normalized (typically the features
axis). For instance, after a
||Momentum for the moving average.|
||Small float added to variance to avoid dividing by zero.|
If True, add offset of
If True, multiply by
||Initializer for the beta weight.|
||Initializer for the gamma weight.|
||Initializer for the moving mean.|
||Initializer for the moving variance.|
||Optional regularizer for the beta weight.|
||Optional regularizer for the gamma weight.|
||Optional constraint for the beta weight.|
||Optional constraint for the gamma weight.|