public class
AdaGradDA
Optimizer that implements the Adagrad Dual-Averaging algorithm.
This optimizer takes care of regularization of unseen features in a mini batch by updating them when they are seen with a closed form update rule that is equivalent to having updated them on every mini-batch.
AdagradDA is typically used when there is a need for large sparsity in the trained model. This optimizer only guarantees sparsity for linear models. Be careful when using AdagradDA for deep networks as it will require careful initialization of the gradient accumulators for it to train.
Constants
| String | ACCUMULATOR | |
| float | INITIAL_ACCUMULATOR_DEFAULT | |
| float | L1_STRENGTH_DEFAULT | |
| float | L2_STRENGTH_DEFAULT | |
| float | LEARNING_RATE_DEFAULT | |
| String | SQUARED_ACCUMULATOR |
Inherited Constants
Public Constructors
Public Methods
| String |
getOptimizerName()
Get the Name of the optimizer.
|
| String |
toString()
|
Inherited Methods
Constants
public static final String ACCUMULATOR
Constant Value:
"gradient_accumulator"
public static final float INITIAL_ACCUMULATOR_DEFAULT
Constant Value:
0.1
public static final float L1_STRENGTH_DEFAULT
Constant Value:
0.0
public static final float L2_STRENGTH_DEFAULT
Constant Value:
0.0
public static final float LEARNING_RATE_DEFAULT
Constant Value:
0.001
public static final String SQUARED_ACCUMULATOR
Constant Value:
"gradient_squared_accumulator"
Public Constructors
public AdaGradDA (Graph graph, float learningRate)
Creates an AdaGradDA Optimizer
Parameters
| graph | the TensorFlow Graph |
|---|---|
| learningRate | the learning rate |
public AdaGradDA (Graph graph, float learningRate, float initialAccumulatorValue, float l1Strength, float l2Strength)
Creates an AdaGradDA Optimizer
Parameters
| graph | the TensorFlow Graph |
|---|---|
| learningRate | the learning rate |
| initialAccumulatorValue | Starting value for the accumulators, must be greater than zero. |
| l1Strength | l1 regularization strength, must be greater than or equal to zero. |
| l2Strength | l2 regularization strength, must be greater than or equal to zero. |
Throws
| IllegalArgumentException | if initialAccumulatorValue is not greater than zero, or l1Strength or l2Strength is less than zero |
|---|
public AdaGradDA (Graph graph, String name, float learningRate)
Creates an AdaGradDA Optimizer
Parameters
| graph | the TensorFlow Graph |
|---|---|
| name | the name for this Optimizer (defaults to 'adagrad-da') |
| learningRate | the learning rate |
public AdaGradDA (Graph graph, String name, float learningRate, float initialAccumulatorValue, float l1Strength, float l2Strength)
Creates an AdaGradDA Optimizer
Parameters
| graph | the TensorFlow Graph |
|---|---|
| name | the name for this Optimizer (defaults to 'adagrad-da') |
| learningRate | the learning rate |
| initialAccumulatorValue | Starting value for the accumulators, must be positive |
| l1Strength | l1 regularization strength, must be greater than or equal to zero. |
| l2Strength | l2 regularization strength, must be greater than or equal to zero. |
Throws
| IllegalArgumentException | if initialAccumulatorValue is not greater than zero, or * l1Strength or l2Strength is less than zero |
|---|
Public Methods
public String getOptimizerName ()
Get the Name of the optimizer.
Returns
- The optimizer name.
public String toString ()