AdaGradDA

public class AdaGradDA

Optimizer that implements the Adagrad Dual-Averaging algorithm.

This optimizer takes care of regularization of unseen features in a mini batch by updating them when they are seen with a closed form update rule that is equivalent to having updated them on every mini-batch.

AdagradDA is typically used when there is a need for large sparsity in the trained model. This optimizer only guarantees sparsity for linear models. Be careful when using AdagradDA for deep networks as it will require careful initialization of the gradient accumulators for it to train.

Constants

String ACCUMULATOR
float INITIAL_ACCUMULATOR_DEFAULT
float L1_STRENGTH_DEFAULT
float L2_STRENGTH_DEFAULT
float LEARNING_RATE_DEFAULT
String SQUARED_ACCUMULATOR

Inherited Constants

Public Constructors

AdaGradDA(Graph graph)
Creates an AdaGradDA Optimizer
AdaGradDA(Graph graph, float learningRate)
Creates an AdaGradDA Optimizer
AdaGradDA(Graph graph, float learningRate, float initialAccumulatorValue, float l1Strength, float l2Strength)
Creates an AdaGradDA Optimizer
AdaGradDA(Graph graph, String name, float learningRate)
Creates an AdaGradDA Optimizer
AdaGradDA(Graph graph, String name, float learningRate, float initialAccumulatorValue, float l1Strength, float l2Strength)
Creates an AdaGradDA Optimizer

Public Methods

String
getOptimizerName()
Get the Name of the optimizer.
String

Inherited Methods

Constants

public static final String ACCUMULATOR

Constant Value: "gradient_accumulator"

public static final float INITIAL_ACCUMULATOR_DEFAULT

Constant Value: 0.1

public static final float L1_STRENGTH_DEFAULT

Constant Value: 0.0

public static final float L2_STRENGTH_DEFAULT

Constant Value: 0.0

public static final float LEARNING_RATE_DEFAULT

Constant Value: 0.001

public static final String SQUARED_ACCUMULATOR

Constant Value: "gradient_squared_accumulator"

Public Constructors

public AdaGradDA (Graph graph)

Creates an AdaGradDA Optimizer

Parameters
graph the TensorFlow Graph

public AdaGradDA (Graph graph, float learningRate)

Creates an AdaGradDA Optimizer

Parameters
graph the TensorFlow Graph
learningRate the learning rate

public AdaGradDA (Graph graph, float learningRate, float initialAccumulatorValue, float l1Strength, float l2Strength)

Creates an AdaGradDA Optimizer

Parameters
graph the TensorFlow Graph
learningRate the learning rate
initialAccumulatorValue Starting value for the accumulators, must be greater than zero.
l1Strength l1 regularization strength, must be greater than or equal to zero.
l2Strength l2 regularization strength, must be greater than or equal to zero.
Throws
IllegalArgumentException if initialAccumulatorValue is not greater than zero, or l1Strength or l2Strength is less than zero

public AdaGradDA (Graph graph, String name, float learningRate)

Creates an AdaGradDA Optimizer

Parameters
graph the TensorFlow Graph
name the name for this Optimizer (defaults to 'adagrad-da')
learningRate the learning rate

public AdaGradDA (Graph graph, String name, float learningRate, float initialAccumulatorValue, float l1Strength, float l2Strength)

Creates an AdaGradDA Optimizer

Parameters
graph the TensorFlow Graph
name the name for this Optimizer (defaults to 'adagrad-da')
learningRate the learning rate
initialAccumulatorValue Starting value for the accumulators, must be positive
l1Strength l1 regularization strength, must be greater than or equal to zero.
l2Strength l2 regularization strength, must be greater than or equal to zero.
Throws
IllegalArgumentException if initialAccumulatorValue is not greater than zero, or * l1Strength or l2Strength is less than zero

Public Methods

public String getOptimizerName ()

Get the Name of the optimizer.

Returns
  • The optimizer name.

public String toString ()