Stay organized with collections Save and categorize content based on your preferences.

View source on GitHub

Initializer that generates tensors without scaling variance.

Inherits From: Initializer

When initializing a deep network, it is in principle advantageous to keep the scale of the input variance constant, so it does not explode or diminish by reaching the final layer. If the input is x and the operation x * W, and we want to initialize W uniformly at random, we need to pick W from

[-sqrt(3) / sqrt(dim), sqrt(3) / sqrt(dim)]

to keep the scale intact, where dim = W.shape[0] (the size of the input). A similar calculation for convolutional networks gives an analogous result with dim equal to the product of the first 3 dimensions. When nonlinearities are present, we need to multiply this by a constant factor. See (Sussillo et al., 2014) for deeper motivation, experiments and the calculation of constants. In section 2.3 there, the constants were numerically computed: for a linear layer it's 1.0, relu: ~1.43, tanh: ~1.15.

factor Float. A multiplicative factor by which the values will be scaled.
seed A Python integer. Used to create random seeds. See tf.compat.v1.set_random_seed for behavior.
dtype Default data type, used if no dtype argument is provided when calling the initializer. Only floating point types are supported.


Sussillo et al., 2014 (pdf)



View source

Instantiates an initializer from a configuration dictionary.


initializer = RandomUniform(-1, 1)
config = initializer.get_config()
initializer = RandomUniform.from_config(config)

config A Python dictionary. It will typically be the output of get_config.

An Initializer instance.


View source

Returns the configuration of the initializer as a JSON-serializable dict.

A JSON-serializable Python dict.


View source

Returns a tensor object initialized as specified by the initializer.

shape Shape of the tensor.
dtype Optional dtype of the tensor. If not provided use the initializer dtype.
partition_info Optional information about the possible partitioning of a tensor.