|TensorFlow 1 version||View source on GitHub|
Maintains moving averages of variables by employing an exponential decay.
Compat aliases for migration
See Migration guide for more details.
tf.train.ExponentialMovingAverage( decay, num_updates=None, zero_debias=False, name='ExponentialMovingAverage' )
When training a model, it is often beneficial to maintain moving averages of the trained parameters. Evaluations that use averaged parameters sometimes produce significantly better results than the final trained values.
apply() method adds shadow copies of trained variables and add ops that
maintain a moving average of the trained variables in their shadow copies.
It is used when building the training model. The ops that maintain moving
averages are typically run after each training step.
average_name() methods give access to the shadow
variables and their names. They are useful when building an evaluation
model, or when restoring a model from a checkpoint file. They help use the
moving averages in place of the last trained values for evaluations.
The moving averages are computed using exponential decay. You specify the
decay value when creating the
ExponentialMovingAverage object. The shadow
variables are initialized with the same initial values as the trained
variables. When you run the ops to maintain the moving averages, each
shadow variable is updated with the formula:
shadow_variable -= (1 - decay) * (shadow_variable - variable)
This is mathematically equivalent to the classic formula below, but the use
assign_sub op (the
"-=" in the formula) allows con