# tf.compat.v1.mixed_precision.DynamicLossScale

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/tensorflow/blob/v2.13.1/tensorflow/python/training/experimental/loss_scale.py#L296-L437) |

Loss scale that dynamically adjusts itself.

Inherits From: [`LossScale`](https://www.tensorflow.org/versions/r2.13/api_docs/python/tf/compat/v1/mixed_precision/LossScale)

#### View aliases


**Compat aliases for migration**

See
[Migration guide](https://www.tensorflow.org/guide/migrate) for
more details.

[`tf.compat.v1.mixed_precision.experimental.DynamicLossScale`](https://www.tensorflow.org/api_docs/python/tf/compat/v1/mixed_precision/DynamicLossScale), [`tf.compat.v1.train.experimental.DynamicLossScale`](https://www.tensorflow.org/api_docs/python/tf/compat/v1/mixed_precision/DynamicLossScale)

<br />

    tf.compat.v1.mixed_precision.DynamicLossScale(
        initial_loss_scale=(2 ** 15), increment_period=2000, multiplier=2.0
    )

Dynamic loss scaling works by adjusting the loss scale as training progresses.
The goal is to keep the loss scale as high as possible without overflowing the
gradients. As long as the gradients do not overflow, raising the loss scale
never hurts.

The algorithm starts by setting the loss scale to an initial value. Every N
steps that the gradients are finite, the loss scale is increased by some
factor. However, if a NaN or Inf gradient is found, the gradients for that
step are not applied, and the loss scale is decreased by the factor. This
process tends to keep the loss scale as high as possible without gradients
overflowing.

<br />

<br />

<br />

| ## Args ||
|---|---|
| `initial_loss_scale` | A Python float. The loss scale to use at the beginning. It's better to start this at a very high number, because a loss scale that is too high gets lowered far more quickly than a loss scale that is too low gets raised. The default is 2 \*\* 15, which is approximately half the maximum float16 value. |
| `increment_period` | Increases loss scale every `increment_period` consecutive steps that finite gradients are encountered. If a nonfinite gradient is encountered, the count is reset back to zero. |
| `multiplier` | The multiplier to use when increasing or decreasing the loss scale. |

<br />

<br />

<br />

<br />

| ## Attributes ||
|---|---|
| `increment_period` | <br /> <br /> |
| `initial_loss_scale` | <br /> <br /> |
| `multiplier` | <br /> <br /> |

<br />

## Methods

### `from_config`

[View source](https://github.com/tensorflow/tensorflow/blob/v2.13.1/tensorflow/python/training/experimental/loss_scale.py#L194-L197)

    @classmethod
    from_config(
        config
    )

Creates the LossScale from its config.

### `get_config`

[View source](https://github.com/tensorflow/tensorflow/blob/v2.13.1/tensorflow/python/training/experimental/loss_scale.py#L432-L437)

    get_config()

Returns the config of this loss scale.

### `update`

[View source](https://github.com/tensorflow/tensorflow/blob/v2.13.1/tensorflow/python/training/experimental/loss_scale.py#L372-L419)

    update(
        grads
    )

Updates loss scale based on if gradients are finite in current step.

### `__call__`

[View source](https://github.com/tensorflow/tensorflow/blob/v2.13.1/tensorflow/python/training/experimental/loss_scale.py#L369-L370)

    __call__()

Returns the current loss scale as a scalar `float32` tensor.