tfl.lattice_lib.laplacian_regularizer

View source on GitHub

Returns Laplacian regularization loss for Lattice layer.

tfl.lattice_lib.laplacian_regularizer(
    weights,
    lattice_sizes,
    l1=0.0,
    l2=0.0
)

Laplacian regularizer penalizes the difference between adjacent vertices in multi-cell lattice (see publication).

Consider a 3 x 2 lattice with weights w:

w[3]-----w[4]-----w[5]
  |        |        |
  |        |        |
w[0]-----w[1]-----w[2]

where the number at each node represents the weight index. In this case, the laplacian regularizer is defined as:

l1[0] * (|w[1] - w[0]| + |w[2] - w[1]| +
         |w[4] - w[3]| + |w[5] - w[4]|) +
l1[1] * (|w[3] - w[0]| + |w[4] - w[1]| + |w[5] - w[2]|) +

l2[0] * ((w[1] - w[0])^2 + (w[2] - w[1])^2 +
         (w[4] - w[3])^2 + (w[5] - w[4])^2) +
l2[1] * ((w[3] - w[0])^2 + (w[4] - w[1])^2 + (w[5] - w[2])^2)

Arguments:

  • weights: Lattice weights tensor of shape: (prod(lattice_sizes), units).
  • lattice_sizes: List or tuple of integers which represents lattice sizes.
  • l1: l1 regularization amount. Either single float or list or tuple of floats to specify different regularization amount per dimension.
  • l2: l2 regularization amount. Either single float or list or tuple of floats to specify different regularization amount per dimension.

Returns:

Laplacian regularization loss.