# SGD

``````public class SGD<Model: Differentiable>: Optimizer
where
Model.TangentVector: VectorProtocol & ElementaryFunctions & KeyPathIterable,
Model.TangentVector.VectorSpaceScalar == Float``````

A stochastic gradient descent (SGD) optimizer.

Implements the stochastic gradient descent algorithm with support for momentum, learning rate decay, and Nesterov momentum. Momentum and Nesterov momentum (a.k.a. the Nesterov accelerated gradient method) are first-order optimization methods that can improve the training speed and convergence rate of gradient descent.

References:

• ``` Model ```

#### Declaration

``public typealias Model = Model``
• ``` learningRate ```

The learning rate.

#### Declaration

``public var learningRate: Float``
• ``` momentum ```

The momentum factor. It accelerates stochastic gradient descent in the relevant direction and dampens oscillations.

#### Declaration

``public var momentum: Float``
• ``` decay ```

The learning rate decay.

#### Declaration

``public var decay: Float``
• ``` nesterov ```

Use Nesterov momentum if true.

#### Declaration

``public var nesterov: Bool``
• ``` velocity ```

The velocity state of the model.

#### Declaration

``public var velocity: Model.TangentVector``
• ``` step ```

The set of steps taken.

#### Declaration

``public var step: Int``
• ``` init(for:learningRate:momentum:decay:nesterov:) ```

Creates an instance for `model`.

#### Declaration

``````public init(
for model: __shared Model,
learningRate: Float = 0.01,
momentum: Float = 0,
decay: Float = 0,
nesterov: Bool = false
)``````

#### Parameters

 ``` learningRate ``` The learning rate. The default value is `0.01`. ``` momentum ``` The momentum factor that accelerates stochastic gradient descent in the relevant direction and dampens oscillations. The default value is `0`. ``` decay ``` The learning rate decay. The default value is `0`. ``` nesterov ``` Use Nesterov momentum iff `true`. The default value is `true`.
• ``` update(_:along:) ```

#### Declaration

``public func update(_ model: inout Model, along direction: Model.TangentVector)``
• ``` init(copying:to:) ```

#### Declaration

``public required init(copying other: SGD, to device: Device)``