Missed TensorFlow Dev Summit? Check out the video playlist. Watch recordings

tfp.experimental.nn.util.make_kernel_bias_prior_spike_and_slab

View source on GitHub

Create prior for Variational layers with kernel and bias.

tfp.experimental.nn.util.make_kernel_bias_prior_spike_and_slab(
    kernel_shape, bias_shape, kernel_initializer=None, bias_initializer=None,
    kernel_batch_ndims=0, bias_batch_ndims=0, dtype=tf.float32,
    kernel_name='prior_kernel', bias_name='prior_bias'
)
scale    = (2. * l2weight)**-0.5
l2weight = scale**-2. / 2.

have a similar regularizing effect.

The std. deviation of each of the component distributions returned by this function is approximately 1415 (or approximately l2weight = 25e-6). In other words this prior is extremely "weak".

Args:

  • kernel_shape: ...
  • bias_shape: ...
  • kernel_initializer: Ignored. Default value: None (i.e., tf.initializers.glorot_uniform()).
  • bias_initializer: Ignored. Default value: None (i.e., tf.zeros).
  • kernel_batch_ndims: ... Default value: 0.
  • bias_batch_ndims: ... Default value: 0.
  • dtype: ... Default value: tf.float32.
  • kernel_name: ... Default value: "prior_kernel".
  • bias_name: ... Default value: "prior_bias".

Returns:

  • kernel_and_bias_distribution: ...