Layer that projects its inputs into a random feature space.

Inherits From: Layer, Module

This layer implements a mapping from input space to a space with output_dim dimensions, which approximates shift-invariant kernels. A kernel function K(x, y) is shift-invariant if K(x, y) == k(x - y) for some function k. Many popular Radial Basis Functions (RBF), including Gaussian and Laplacian kernels, are shift-invariant.

The implementation of this layer is based on the following paper: "Random Features for Large-Scale Kernel Machines" by Ali Rahimi and Ben Recht.

The distribution from which the parameters of the random features map (layer) are sampled determines which shift-invariant kernel the layer approximates (see paper for more details). You can use the distribution of your choice. The layer supports out-of-the-box approximation sof the following two RBF kernels:

  • Gaussian: K(x, y) == exp(- square(x - y) / (2 * square(scale)))
  • Laplacian: K(x, y) = exp(-abs(x - y) / scale))