tfm.vision.layers.Scale

Scales the input by a trainable scalar weight.

tfm.vision.layers.Scale(
    initializer: tf.keras.initializers.Initializer = 'ones',
    regularizer: Optional[tf.keras.regularizers.Regularizer] = None,
    **kwargs
)

This is useful for applying ReZero to layers, which improves convergence speed. This implements the paper: ReZero is All You Need: Fast Convergence at Large Depth. (https://arxiv.org/pdf/2003.04887.pdf).

initializer A str of initializer for the scalar weight.
regularizer A tf.keras.regularizers.Regularizer for the scalar weight.
**kwargs Additional keyword arguments to be passed to this layer.

Methods

call

View source

call(
    inputs
)

Calls the layer with the given inputs.