![]() |
Scales the input by a trainable scalar weight.
tfm.vision.layers.Scale(
initializer: tf.keras.initializers.Initializer = 'ones',
regularizer: Optional[tf.keras.regularizers.Regularizer] = None,
**kwargs
)
This is useful for applying ReZero to layers, which improves convergence speed. This implements the paper: ReZero is All You Need: Fast Convergence at Large Depth. (https://arxiv.org/pdf/2003.04887.pdf).
Args | |
---|---|
initializer
|
A str of initializer for the scalar weight.
|
regularizer
|
A tf.keras.regularizers.Regularizer for the scalar weight.
|
**kwargs
|
Additional keyword arguments to be passed to this layer. |
Methods
call
call(
inputs
)
Calls the layer with the given inputs.