public struct Dense<Scalar> : Layer where Scalar : TensorFlowFloatingPoint

A densely-connected neural network layer.

Dense implements the operation activation(matmul(input, weight) + bias), where weight is a weight matrix, bias is a bias vector, and activation is an element-wise activation function.

  • The weight matrix.

    Declaration

    public var weight: Tensor<Scalar>
  • The bias vector.

    Declaration

    public var bias: Tensor<Scalar>
  • Declaration

    public typealias Activation = (Tensor<Scalar>) -> Tensor<Scalar>
  • The element-wise activation function.

    Declaration

    public let activation: Activation
  • Declaration

    public init(
        weight: Tensor<Scalar>,
        bias: Tensor<Scalar>,
        activation: @escaping Activation
    )
  • Returns the output obtained from applying the layer to the given input.

    Declaration

    public func applied(to input: Tensor<Scalar>, in _: Context) -> Tensor<Scalar>

    Parameters

    input

    The input to the layer.

    context

    The contextual information for the layer application, e.g. the current learning phase.

    Return Value

    The output.

  • Creates a Dense layer with the specified input size, output size, and element-wise activation function. The weight matrix is created with shape [inputSize, outputSize] and is initialized using Glorot uniform initialization with the specified generator. The bias vector is created with shape [outputSize] and is initialized with zeros.

    Note

    Use init(inputSize:outputSize:activation:seed:) for faster random initialization.

    Declaration

    init<G: RandomNumberGenerator>(
        inputSize: Int,
        outputSize: Int,
        activation: @escaping Activation = identity,
        generator: inout G
    )

    Parameters

    inputSize

    The dimensionality of the input space.

    outputSize

    The dimensionality of the output space.

    activation

    The activation function to use. The default value is identity(_:).

    generator

    The random number generator for initialization.

  • Declaration

    init(inputSize: Int, outputSize: Int, activation: @escaping Activation = identity)
  • Creates a Dense layer with the specified input size, output size, and element-wise activation function. The weight matrix is created with shape [inputSize, outputSize] and is initialized using Glorot uniform initialization with the specified seed. The bias vector is created with shape [outputSize] and is initialized with zeros.

    Declaration

    init(
        inputSize: Int,
        outputSize: Int,
        activation: @escaping Activation = identity,
        seed: (Int64, Int64) = (Int64.random(in: Int64.min..<Int64.max),
                                Int64.random(in: Int64.min..<Int64.max))
    )

    Parameters

    inputSize

    The dimensionality of the input space.

    outputSize

    The dimensionality of the output space.

    activation

    The activation function to use. The default value is identity(_:).

    seed

    The random seed for initialization. The default value is random.