# tf_privacy.DPSequential

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/privacy/blob/v0.9.0/privacy/privacy/keras_models/dp_keras_model.py#L28-L326) |

DP subclass of [`tf.keras.Sequential`](https://www.tensorflow.org/api_docs/python/tf/keras/Sequential).

    tf_privacy.DPSequential(
        l2_norm_clip,
        noise_multiplier,
        num_microbatches=None,
        use_xla=True,
        layer_registry=None,
        *args,
        **kwargs
    )

This can be used as a differentially private replacement for
tf.keras.Sequential. This class implements DP-SGD using the standard
Gaussian mechanism.

This class also utilizes a faster gradient clipping algorithm if the
following two conditions hold:
(i) the trainable layers of the model are keys in the input
`layer_registry`,
(ii) the loss [`tf.Tensor`](https://www.tensorflow.org/api_docs/python/tf/Tensor) for a given batch of examples is either a
scalar or a 2D [`tf.Tensor`](https://www.tensorflow.org/api_docs/python/tf/Tensor) that has only one column
`(i.e., tf.shape(loss)[1] == 1)` and whose i-th row corresponds to
the loss of the i-th example.
This clipping algorithm specifically computes clipped gradients at the
per-example or per microbatch (when `num_microbatches` is not None)
level using the layer registry functions in `layer_registry` (see
clip_grads.py for more information about the algorithm).

> [!WARNING]
> **Warning:** with faster gradient clipping, and when num_microbatches is not None, the per microbatch loss is assumed to be computed as the mean of the loss over the microbatch, or effectively, by reshaping the loss from the shape \[batch_size, ...\] to the shape \[num_microbatches, batch_size/num_microbatches, ...\] and computing the mean of the loss over the microbatches. This would require that the loss function behaves accordingly. This is true for multiple common predefined keras loss functions (e.g. mean_squared_loss, binary_crossentropy) but may not hold for custom losses (and how such aggregation is done is not exposed by the loss function, unfortunately). It is the caller's responsibility to make sure that the loss function does behave this way.

> [!WARNING]
> **Warning:** This API does not have privacy guarantees for custom layer-level losses created by the `layer.add_loss()` API. It does, however, support layer regularization losses. All of these layer-level losses are found in `model.losses`.

When instantiating this class, you need to supply several
DP-related arguments followed by the standard arguments for
`Sequential`.

Examples:

    # Create Model instance.
    model = DPSequential(l2_norm_clip=1.0, noise_multiplier=0.5, use_xla=True,
             <standard arguments>)

You should use your DPSequential instance with a standard instance
of `tf.keras.Optimizer` as the optimizer, and a standard reduced loss.
You do not need to use a differentially private optimizer.

    # Use a standard (non-DP) optimizer.
    optimizer = tf.keras.optimizers.SGD(learning_rate=0.01)

    # Use a standard reduced loss.
    loss = tf.keras.losses.MeanSquaredError()

    model.compile(optimizer=optimizer, loss=loss)
    model.fit(train_data, train_labels, epochs=1, batch_size=32)

<br />

<br />

<br />

| ## Args ||
|---|---|
| `l2_norm_clip` | Clipping norm (max L2 norm of per microbatch gradients). |
| `noise_multiplier` | Ratio of the standard deviation to the clipping norm. |
| `num_microbatches` | Number of microbatches. |
| `use_xla` | If `True`, compiles train_step to XLA. |
| `layer_registry` | A `LayerRegistry` instance containing functions that help compute gradient norms quickly. See `tensorflow_privacy.privacy.fast_gradient_clipping.layer_registry` for more details. |
| `*args` | These will be passed on to the base class `__init__` method. |
| `**kwargs` | These will be passed on to the base class `__init__` method. |

<br />

## Methods

### `add`

    add(
        layer
    )

Adds a layer instance on top of the layer stack.

<br />

<br />

<br />

| Args ||
|---|---|
| `layer` | layer instance. |

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `TypeError` | If `layer` is not a layer instance. |
| `ValueError` | In case the `layer` argument does not know its input shape. |
| `ValueError` | In case the `layer` argument has multiple output tensors, or is already connected somewhere else (forbidden in `Sequential` models). |

<br />

### `pop`

    pop()

Removes the last layer in the model.

<br />

<br />

<br />

| Raises ||
|---|---|
| `TypeError` | if there are no layers in the model. |

<br />