SdcaOptimizer

public final class SdcaOptimizer

Distributed version of Stochastic Dual Coordinate Ascent (SDCA) optimizer for

linear models with L1 + L2 regularization. As global optimization objective is strongly-convex, the optimizer optimizes the dual objective at each step. The optimizer applies each update one example at a time. Examples are sampled uniformly, and the optimizer is learning rate free and enjoys linear convergence rate.

[Proximal Stochastic Dual Coordinate Ascent](http://arxiv.org/pdf/1211.2717v1.pdf).
Shai Shalev-Shwartz, Tong Zhang. 2012

$$Loss Objective = \sum f_{i} (wx_{i}) + (l2 / 2) * |w|^2 + l1 * |w|$$

[Adding vs. Averaging in Distributed Primal-Dual Optimization](http://arxiv.org/abs/1502.03508).
Chenxin Ma, Virginia Smith, Martin Jaggi, Michael I. Jordan, Peter Richtarik, Martin Takac. 2015

[Stochastic Dual Coordinate Ascent with Adaptive Probabilities](https://arxiv.org/abs/1502.08053).
Dominik Csiba, Zheng Qu, Peter Richtarik. 2015

Nested Classes

class SdcaOptimizer.Options Optional attributes for SdcaOptimizer

Constants

String OP_NAME The name of this op, as known by TensorFlow core engine

Public Methods

static SdcaOptimizer.Options
adaptive (Boolean adaptive)
static SdcaOptimizer
create ( Scope scope, Iterable< Operand < TInt64 >> sparseExampleIndices, Iterable< Operand < TInt64 >> sparseFeatureIndices, Iterable< Operand < TFloat32 >> sparseFeatureValues, Iterable< Operand < TFloat32 >> denseFeatures, Operand < TFloat32 > exampleWeights, Operand < TFloat32 > exampleLabels, Iterable< Operand < TInt64 >> sparseIndices, Iterable< Operand < TFloat32 >> sparseWeights, Iterable< Operand < TFloat32 >> denseWeights, Operand < TFloat32 > exampleStateData, String lossType, Float l1, Float l2, Long numLossPartitions, Long numInnerIterations, Options... options)
Factory method to create a class wrapping a new SdcaOptimizer operation.
List< Output < TFloat32 >>
outDeltaDenseWeights ()
a list of vectors where the values are the delta weights associated with a dense feature group.
List< Output < TFloat32 >>
outDeltaSparseWeights ()
a list of vectors where each value is the delta weights associated with a sparse feature group.
Output < TFloat32 >
outExampleStateData ()
a list of vectors containing the updated example state data.

Inherited Methods

Constants

public static final String OP_NAME

The name of this op, as known by TensorFlow core engine

Constant Value: "SdcaOptimizerV2"

Public Methods

public static SdcaOptimizer.Options adaptive (Boolean adaptive)

Parameters
adaptive Whether to use Adaptive SDCA for the inner loop.

public static SdcaOptimizer create ( Scope scope, Iterable< Operand < TInt64 >> sparseExampleIndices, Iterable< Operand < TInt64 >> sparseFeatureIndices, Iterable< Operand < TFloat32 >> sparseFeatureValues, Iterable< Operand < TFloat32 >> denseFeatures, Operand < TFloat32 > exampleWeights, Operand < TFloat32 > exampleLabels, Iterable< Operand < TInt64 >> sparseIndices, Iterable< Operand < TFloat32 >> sparseWeights, Iterable< Operand < TFloat32 >> denseWeights, Operand < TFloat32 > exampleStateData, String lossType, Float l1, Float l2, Long numLossPartitions, Long numInnerIterations, Options... options)

Factory method to create a class wrapping a new SdcaOptimizer operation.

Parameters
scope current scope
sparseExampleIndices a list of vectors which contain example indices.
sparseFeatureIndices a list of vectors which contain feature indices.
sparseFeatureValues a list of vectors which contains feature value associated with each feature group.
denseFeatures a list of matrices which contains the dense feature values.
exampleWeights a vector which contains the weight associated with each example.
exampleLabels a vector which contains the label/target associated with each example.
sparseIndices a list of vectors where each value is the indices which has corresponding weights in sparse_weights. This field maybe omitted for the dense approach.
sparseWeights a list of vectors where each value is the weight associated with a sparse feature group.
denseWeights a list of vectors where the values are the weights associated with a dense feature group.
exampleStateData a list of vectors containing the example state data.
lossType Type of the primal loss. Currently SdcaSolver supports logistic, squared and hinge losses.
l1 Symmetric l1 regularization strength.
l2 Symmetric l2 regularization strength.
numLossPartitions Number of partitions of the global loss function.
numInnerIterations Number of iterations per mini-batch.
options carries optional attributes values
Returns
  • a new instance of SdcaOptimizer

public List< Output < TFloat32 >> outDeltaDenseWeights ()

a list of vectors where the values are the delta weights associated with a dense feature group.

public List< Output < TFloat32 >> outDeltaSparseWeights ()

a list of vectors where each value is the delta weights associated with a sparse feature group.

public Output < TFloat32 > outExampleStateData ()

a list of vectors containing the updated example state data.