tf.keras.initializers.VarianceScaling

View source on GitHub

Initializer capable of adapting its scale to the shape of weights tensors.

Inherits From: Initializer

tf.keras.initializers.VarianceScaling(
    scale=1.0, mode='fan_in', distribution='truncated_normal', seed=None
)

Initializers allow you to pre-specify an initialization strategy, encoded in the Initializer object, without knowing the shape and dtype of the variable being initialized.

With distribution="truncated_normal" or "untruncated_normal", samples are drawn from a truncated/untruncated normal distribution with a mean of zero and a standard deviation (after truncation, if used) stddev = sqrt(scale / n) where n is:

  • number of input units in the weight tensor, if mode = "fan_in"
  • number of output units, if mode = "fan_out"
  • average of the numbers of input and output units, if mode = "fan_avg"

With distribution="uniform", samples are drawn from a uniform distribution within [-limit, limit], with limit = sqrt(3 * scale / n).

Examples:

def make_variables(k, initializer): 
  return (tf.Variable(initializer(shape=[k], dtype=tf.float32)), 
          tf.Variable(initializer(shape=[k, k], dtype=tf.float32))) 
v1, v2 = make_variables(3, tf.initializers.VarianceScaling(scale=1.)) 
v1 
<tf.Variable ... shape=(3,) ... numpy=array([...], dtype=float32)> 
v2 
<tf.Variable ... shape=(3, 3) ... numpy= 
 
make_variables(4, tf.initializers.VarianceScaling(distribution='uniform')) 
(<tf.Variable...shape=(4,) dtype=float32...>, <tf.Variable...shape=(4, 4) ... 

Args:

  • scale: Scaling factor (positive float).
  • mode: One of "fan_in", "fan_out", "fan_avg".
  • distribution: Random distribution to use. One of "truncated_normal", "untruncated_normal" and "uniform".
  • seed: A Python integer. Used to create random seeds. See tf.random.set_seed for behavior.

Raises:

  • ValueError: In case of an invalid value for the "scale", mode" or "distribution" arguments.

Methods

__call__

View source

__call__(
    shape, dtype=tf.dtypes.float32
)

Returns a tensor object initialized as specified by the initializer.

Args:

  • shape: Shape of the tensor.
  • dtype: Optional dtype of the tensor. Only floating point types are supported.

Raises:

  • ValueError: If the dtype is not floating point

from_config

View source

@classmethod
from_config(
    config
)

Instantiates an initializer from a configuration dictionary.

Example:

initializer = RandomUniform(-1, 1)
config = initializer.get_config()
initializer = RandomUniform.from_config(config)

Args:

  • config: A Python dictionary. It will typically be the output of get_config.

Returns:

An Initializer instance.

get_config

View source

get_config()

Returns the configuration of the initializer as a JSON-serializable dict.

Returns:

A JSON-serializable Python dict.