Classes

The following classes are available globally.


                    
                    
                    Parameter

A mutable, shareable, owning reference to a tensor.

Declaration

public final class Parameter<Scalar> where Scalar : TensorFlowScalar

TFETensorHandle
Class wrapping a C pointer to a TensorHandle. This class owns the TensorHandle and is responsible for destroying it.
Declaration
public class TFETensorHandle : _AnyTensorHandle

RMSProp
RMSProp optimizer.

It is recommended to leave the parameters of this optimizer at their default values (except for the learning rate, which can be freely tuned). This optimizer is usually a good choice for recurrent neural networks.

Reference: rmsprop: Divide the gradient by a running average of its recent magnitude
Declaration
public class RMSProp<Model: Differentiable>: Optimizer where Model.TangentVector: VectorProtocol & PointwiseMultiplicative & ElementaryFunctions, Model.TangentVector.VectorSpaceScalar == Float
AdaGrad
AdaGrad optimizer.

Individually adapts the learning rates of all model parameters by scaling them inversely proportional to the square root of the sum of all the historical squared values of the gradient.

Reference: Adaptive Subgradient Methods for Online Learning and Stochastic Optimization
Declaration
public class AdaGrad<Model: Differentiable>: Optimizer where Model.TangentVector: VectorProtocol & PointwiseMultiplicative & ElementaryFunctions, Model.TangentVector.VectorSpaceScalar == Float
AdaDelta
ADADELTA optimizer.

ADADELTA is a more robust extension of AdaGrad. ADADELTA adapts learning rates based on a moving window of gradient updates rather than by accumulating all past gradient norms. It can thus adapt faster to changing dynamics of the optimization problem space.

Reference: ADADELTA: An Adaptive Learning Rate Method
Declaration
public class AdaDelta<Model: Differentiable>: Optimizer where Model.TangentVector: VectorProtocol & PointwiseMultiplicative & ElementaryFunctions, Model.TangentVector.VectorSpaceScalar == Float


                    
                    
                    Adam

Adam optimizer.

Reference: Adam - A Method for Stochastic Optimization

Declaration

public class Adam<Model: Differentiable>: Optimizer
    where Model.TangentVector: VectorProtocol & PointwiseMultiplicative & ElementaryFunctions,
          Model.TangentVector.VectorSpaceScalar == Float


                    
                    
                    AdaMax

AdaMax optimizer.

A variant of Adam based on the infinity-norm.

Reference: Section 7 of Adam - A Method for Stochastic Optimization

Declaration

public class AdaMax<Model: Differentiable & KeyPathIterable>: Optimizer
    where Model.TangentVector: VectorProtocol & PointwiseMultiplicative & 
                               ElementaryFunctions & KeyPathIterable,
          Model.TangentVector.VectorSpaceScalar == Float


                    
                    
                    AMSGrad

AMSGrad optimizer.

This algorithm is a modification of Adam with better convergence properties when close to local optima.

Reference: On the Convergence of Adam and Beyond

Declaration

public class AMSGrad<Model: Differentiable & KeyPathIterable>: Optimizer
    where Model.TangentVector: VectorProtocol & PointwiseMultiplicative & 
                               ElementaryFunctions & KeyPathIterable,
          Model.TangentVector.VectorSpaceScalar == Float

SGD

Stochastic gradient descent (SGD) optimizer.

An optimizer that implements stochastic gradient descent, with support for momentum, learning rate decay, and Nesterov momentum.

Declaration

public class SGD<Model: Differentiable>: Optimizer
    where Model.TangentVector: VectorProtocol & ElementaryFunctions,
          Model.TangentVector.VectorSpaceScalar == Float