Differentiable

public protocol Differentiable

A type that mathematically represents a differentiable manifold whose tangent spaces are finite-dimensional.

  • A type representing a differentiable value’s derivatives.

    Mathematically, this is equivalent to the tangent bundle of the differentiable manifold represented by the differentiable type.

    Declaration

    associatedtype TangentVector: Differentiable & AdditiveArithmetic
      where TangentVector.TangentVector == TangentVector
  • Moves self along the given direction. In Riemannian geometry, this is equivalent to exponential map, which moves self on the geodesic surface along the given tangent vector.

    Declaration

    mutating mutating func move(along direction: TangentVector)
  • zeroTangentVectorInitializer

    Default implementation

    A closure that produces a zero tangent vector, capturing minimal necessary information from self.

    move(along: zeroTangentVectorInitializer()) should not modify self.

    In some cases, the zero tangent vector of self is equal to TangentVector.zero. In other cases, the zero tangent vector depends on information in self, such as shape for an n-dimensional array type. For differentiable programming, it is more memory-efficient to define a custom zeroTangentVectorInitializer property which returns a closure that captures and uses only the necessary information to create a zero tangent vector. For example:

    struct Vector {
        var scalars: [Float]
        var count: Int { scalars.count }
        init(scalars: [Float]) { ... }
        init(repeating repeatedElement: Float, count: Int) { ... }
    }
    
    extension Vector: AdditiveArithmetic { ... }
    
    extension Vector: Differentiable {
        typealias TangentVector = Vector
    
        @noDerivative
        var zeroTangentVectorInitializer: () -> TangentVector {
            let count = self.count
            return { TangentVector(repeating: 0, count: count) }
        }
    }
    

    Default Implementation

    Declaration

    var zeroTangentVectorInitializer: () -> TangentVector { get }
  • zeroTangentVector

    Extension method

    A tangent vector initialized using zeroTangentVectorInitializer. move(along: zeroTangentVector) should not modify self.

    Declaration

    var zeroTangentVector: TangentVector { get }
  • Declaration

    @differentiable(wrt: self)
    func withRecomputationInPullbacks<Result : Differentiable>(
      _ body: @escaping @differentiable (Self) -> Result
    ) -> Result
  • withDerivative(_:)

    Extension method

    Applies the given closure to the derivative of self.

    Returns self like an identity function. When the return value is used in a context where it is differentiated with respect to, applies the given closure to the derivative of the return value.

    Declaration

    @differentiable(wrt: self)
    func withDerivative(_ body: @escaping (inout TangentVector) -> Void) -> Self
  • sequenced(through:_:)

    Extension method

    Returns the output computed by applying a sequence of layers to the previous layer’s output, except that the first layer’s input is self.

    Declaration

    @differentiable
    public func sequenced<L1: Layer, L2: Layer>(through l1: L1, _ l2: L2) -> L2.Output
    where L1.Input == Self, L1.Output == L2.Input

    Parameters

    l1

    The first layer.

    l2

    The second layer.

    Return Value

    The final layer’s output after sequential application.

  • sequenced(through:_:_:)

    Extension method

    Returns the output computed by applying a sequence of layers to the previous layer’s output, except that the first layer’s input is self.

    Declaration

    @differentiable
    public func sequenced<L1: Layer, L2: Layer, L3: Layer>(through l1: L1, _ l2: L2, _ l3: L3)
      -> L3.Output
    where L1.Input == Self, L1.Output == L2.Input, L2.Output == L3.Input

    Parameters

    l1

    The first layer.

    l2

    The second layer.

    l3

    The third layer.

    Return Value

    The final layer’s output after sequential application.

  • sequenced(through:_:_:_:)

    Extension method

    Returns the output computed by applying a sequence of layers to the previous layer’s output, except that the first layer’s input is self.

    Declaration

    @differentiable
    public func sequenced<L1: Layer, L2: Layer, L3: Layer, L4: Layer>(
      through l1: L1, _ l2: L2, _ l3: L3, _ l4: L4
    ) -> L4.Output
    where
      L1.Input == Self, L1.Output == L2.Input, L2.Output == L3.Input,
      L3.Output == L4.Input

    Parameters

    l1

    The first layer.

    l2

    The second layer.

    l3

    The third layer.

    l4

    The fourth layer.

    Return Value

    The final layer’s output after sequential application.

  • sequenced(through:_:_:_:_:)

    Extension method

    Returns the output computed by applying a sequence of layers to the previous layer’s output, except that the first layer’s input is self.

    Declaration

    @differentiable
    public func sequenced<L1: Layer, L2: Layer, L3: Layer, L4: Layer, L5: Layer>(
      through l1: L1, _ l2: L2, _ l3: L3, _ l4: L4, _ l5: L5
    ) -> L5.Output
    where
      L1.Input == Self, L1.Output == L2.Input, L2.Output == L3.Input, L3.Output == L4.Input,
      L4.Output == L5.Input

    Parameters

    l1

    The first layer.

    l2

    The second layer.

    l3

    The third layer.

    l4

    The third layer.

    l5

    The fifth layer.

    Return Value

    The final layer’s output after sequential application.

  • Returns the output computed by applying a sequence of layers to the previous layer’s output, except that the first layer’s input is self.

    Declaration

    @differentiable
    public func sequenced<L1: Layer, L2: Layer, L3: Layer, L4: Layer, L5: Layer, L6: Layer>(
      through l1: L1, _ l2: L2, _ l3: L3, _ l4: L4, _ l5: L5, _ l6: L6
    ) -> L6.Output
    where
      L1.Input == Self, L1.Output == L2.Input, L2.Output == L3.Input, L3.Output == L4.Input,
      L4.Output == L5.Input, L5.Output == L6.Input

    Parameters

    l1

    The first layer.

    l2

    The second layer.

    l3

    The third layer.

    l4

    The third layer.

    l5

    The fifth layer.

    l6

    The sixth layer.

    Return Value

    The final layer’s output after sequential application.