Assess privacy risks with the TensorFlow Privacy Report

View on TensorFlow.org Run in Google Colab View source on GitHub Download notebook

Overview

In this codelab you'll train a simple image classification model on the CIFAR10 dataset, and then use the "membership inference attack" against this model to assess if the attacker is able to "guess" whether a particular sample was present in the training set. You will use the TF Privacy Report to visualize results from multiple models and model checkpoints.

Setup

import numpy as np
from typing import Tuple
from scipy import special
from sklearn import metrics

import tensorflow as tf

import tensorflow_datasets as tfds

# Set verbosity.
tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)
from sklearn.exceptions import ConvergenceWarning

import warnings
warnings.simplefilter(action="ignore", category=ConvergenceWarning)
warnings.simplefilter(action="ignore", category=FutureWarning)

Install TensorFlow Privacy.

pip install tensorflow_privacy
from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack import membership_inference_attack as mia
from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack.data_structures import AttackInputData
from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack.data_structures import AttackResultsCollection
from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack.data_structures import AttackType
from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack.data_structures import PrivacyMetric
from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack.data_structures import PrivacyReportMetadata
from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack.data_structures import SlicingSpec
from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack import privacy_report
import tensorflow_privacy

Train two models, with privacy metrics

This section trains a pair of keras.Model classifiers on the CIFAR-10 dataset. During the training process it collects privacy metrics, that will be used to generate reports in the bext section.

The first step is to define some hyperparameters:

dataset = 'cifar10'
num_classes = 10
activation = 'relu'
num_conv = 3

batch_size=50
epochs_per_report = 2
total_epochs = 50

lr = 0.001

Next, load the dataset. There's nothing privacy-specific in this code.

print('Loading the dataset.')
train_ds = tfds.as_numpy(
    tfds.load(dataset, split=tfds.Split.TRAIN, batch_size=-1))
test_ds = tfds.as_numpy(
    tfds.load(dataset, split=tfds.Split.TEST, batch_size=-1))
x_train = train_ds['image'].astype('float32') / 255.
y_train_indices = train_ds['label'][:, np.newaxis]
x_test = test_ds['image'].astype('float32') / 255.
y_test_indices = test_ds['label'][:, np.newaxis]

# Convert class vectors to binary class matrices.
y_train = tf.keras.utils.to_categorical(y_train_indices, num_classes)
y_test = tf.keras.utils.to_categorical(y_test_indices, num_classes)

input_shape = x_train.shape[1:]

assert x_train.shape[0] % batch_size == 0, "The tensorflow_privacy optimizer doesn't handle partial batches"
Loading the dataset.
2021-08-20 22:16:52.159316: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-20 22:16:52.167694: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-20 22:16:52.168581: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-20 22:16:52.170437: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-08-20 22:16:52.170991: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-20 22:16:52.171990: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-20 22:16:52.172842: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-20 22:16:52.748161: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-20 22:16:52.749141: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-20 22:16:52.750089: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-20 22:16:52.750900: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory:  -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0
2021-08-20 22:16:53.082253: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)

Next define a function to build the models.

def small_cnn(input_shape: Tuple[int],
              num_classes: int,
              num_conv: int,
              activation: str = 'relu') -> tf.keras.models.Sequential:
  """Setup a small CNN for image classification.

  Args:
    input_shape: Integer tuple for the shape of the images.
    num_classes: Number of prediction classes.
    num_conv: Number of convolutional layers.
    activation: The activation function to use for conv and dense layers.

  Returns:
    The Keras model.
  """
  model = tf.keras.models.Sequential()
  model.add(tf.keras.layers.Input(shape=input_shape))

  # Conv layers
  for _ in range(num_conv):
    model.add(tf.keras.layers.Conv2D(32, (3, 3), activation=activation))
    model.add(tf.keras.layers.MaxPooling2D())

  model.add(tf.keras.layers.Flatten())
  model.add(tf.keras.layers.Dense(64, activation=activation))
  model.add(tf.keras.layers.Dense(num_classes))

  model.compile(
    loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True),
    optimizer=tf.keras.optimizers.Adam(learning_rate=lr),
    metrics=['accuracy'])

  return model

Build two three-layer CNN models using that function.

Configure the first to use a basic SGD optimizer, an the second to use a differentially private optimizer (tf_privacy.DPKerasAdamOptimizer), so you can compare the results.

model_2layers = small_cnn(
    input_shape, num_classes, num_conv=2, activation=activation)
model_3layers = small_cnn(
    input_shape, num_classes, num_conv=3, activation=activation)

Define a callback to collect privacy metrics

Next define a keras.callbacks.Callback to periorically run some privacy attacks against the model, and log the results.

The keras fit method will call the on_epoch_end method after each training epoch. The n argument is the (0-based) epoch number.

You could implement this procedure by writing a loop that repeatedly calls Model.fit(..., epochs=epochs_per_report) and runs the attack code. The callback is used here just because it gives a clear separation between the training logic, and the privacy evaluation logic.

class PrivacyMetrics(tf.keras.callbacks.Callback):
  def __init__(self, epochs_per_report, model_name):
    self.epochs_per_report = epochs_per_report
    self.model_name = model_name
    self.attack_results = []

  def on_epoch_end(self, epoch, logs=None):
    epoch = epoch+1

    if epoch % self.epochs_per_report != 0:
      return

    print(f'\nRunning privacy report for epoch: {epoch}\n')

    logits_train = self.model.predict(x_train, batch_size=batch_size)
    logits_test = self.model.predict(x_test, batch_size=batch_size)

    prob_train = special.softmax(logits_train, axis=1)
    prob_test = special.softmax(logits_test, axis=1)

    # Add metadata to generate a privacy report.
    privacy_report_metadata = PrivacyReportMetadata(
        # Show the validation accuracy on the plot
        # It's what you send to train_accuracy that gets plotted.
        accuracy_train=logs['val_accuracy'], 
        accuracy_test=logs['val_accuracy'],
        epoch_num=epoch,
        model_variant_label=self.model_name)

    attack_results = mia.run_attacks(
        AttackInputData(
            labels_train=y_train_indices[:, 0],
            labels_test=y_test_indices[:, 0],
            probs_train=prob_train,
            probs_test=prob_test),
        SlicingSpec(entire_dataset=True, by_class=True),
        attack_types=(AttackType.THRESHOLD_ATTACK,
                      AttackType.LOGISTIC_REGRESSION),
        privacy_report_metadata=privacy_report_metadata)

    self.attack_results.append(attack_results)

Train the models

The next code block trains the two models. The all_reports list is used to collect all the results from all the models' training runs. The individual reports are tagged witht the model_name, so there's no confusion about which model generated which report.

all_reports = []
callback = PrivacyMetrics(epochs_per_report, "2 Layers")
history = model_2layers.fit(
      x_train,
      y_train,
      batch_size=batch_size,
      epochs=total_epochs,
      validation_data=(x_test, y_test),
      callbacks=[callback],
      shuffle=True)

all_reports.extend(callback.attack_results)
Epoch 1/50
2021-08-20 22:17:10.514876: I tensorflow/stream_executor/cuda/cuda_dnn.cc:369] Loaded cuDNN version 8100
2021-08-20 22:17:15.414683: I tensorflow/core/platform/default/subprocess.cc:304] Start cannot spawn child process: No such file or directory
1000/1000 [==============================] - 12s 3ms/step - loss: 1.5303 - accuracy: 0.4503 - val_loss: 1.3005 - val_accuracy: 0.5390
Epoch 2/50
1000/1000 [==============================] - 3s 3ms/step - loss: 1.1994 - accuracy: 0.5780 - val_loss: 1.1335 - val_accuracy: 0.5995

Running privacy report for epoch: 2

Epoch 3/50
1000/1000 [==============================] - 3s 3ms/step - loss: 1.0646 - accuracy: 0.6246 - val_loss: 1.0789 - val_accuracy: 0.6194
Epoch 4/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.9818 - accuracy: 0.6567 - val_loss: 0.9926 - val_accuracy: 0.6603

Running privacy report for epoch: 4

Epoch 5/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.9181 - accuracy: 0.6787 - val_loss: 0.9919 - val_accuracy: 0.6597
Epoch 6/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.8613 - accuracy: 0.7007 - val_loss: 0.9757 - val_accuracy: 0.6679

Running privacy report for epoch: 6

Epoch 7/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.8179 - accuracy: 0.7168 - val_loss: 0.9535 - val_accuracy: 0.6784
Epoch 8/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.7728 - accuracy: 0.7326 - val_loss: 0.9381 - val_accuracy: 0.6848

Running privacy report for epoch: 8

Epoch 9/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.7413 - accuracy: 0.7417 - val_loss: 0.9557 - val_accuracy: 0.6765
Epoch 10/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.7116 - accuracy: 0.7540 - val_loss: 0.9354 - val_accuracy: 0.6945

Running privacy report for epoch: 10

Epoch 11/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.6804 - accuracy: 0.7627 - val_loss: 0.9587 - val_accuracy: 0.6748
Epoch 12/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.6508 - accuracy: 0.7738 - val_loss: 0.9439 - val_accuracy: 0.6918

Running privacy report for epoch: 12

Epoch 13/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.6255 - accuracy: 0.7825 - val_loss: 0.9705 - val_accuracy: 0.6833
Epoch 14/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.6009 - accuracy: 0.7893 - val_loss: 0.9772 - val_accuracy: 0.6871

Running privacy report for epoch: 14

Epoch 15/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.5798 - accuracy: 0.7980 - val_loss: 0.9561 - val_accuracy: 0.6974
Epoch 16/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.5548 - accuracy: 0.8044 - val_loss: 0.9919 - val_accuracy: 0.6933

Running privacy report for epoch: 16

Epoch 17/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.5306 - accuracy: 0.8143 - val_loss: 1.0101 - val_accuracy: 0.6904
Epoch 18/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.5132 - accuracy: 0.8190 - val_loss: 1.0677 - val_accuracy: 0.6930

Running privacy report for epoch: 18

Epoch 19/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.4902 - accuracy: 0.8277 - val_loss: 1.0380 - val_accuracy: 0.6913
Epoch 20/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.4713 - accuracy: 0.8336 - val_loss: 1.0710 - val_accuracy: 0.6865

Running privacy report for epoch: 20

Epoch 21/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.4548 - accuracy: 0.8398 - val_loss: 1.0892 - val_accuracy: 0.6898
Epoch 22/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.4365 - accuracy: 0.8439 - val_loss: 1.1452 - val_accuracy: 0.6846

Running privacy report for epoch: 22

Epoch 23/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.4228 - accuracy: 0.8514 - val_loss: 1.1787 - val_accuracy: 0.6802
Epoch 24/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.4065 - accuracy: 0.8561 - val_loss: 1.1922 - val_accuracy: 0.6860

Running privacy report for epoch: 24

Epoch 25/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.3909 - accuracy: 0.8607 - val_loss: 1.2336 - val_accuracy: 0.6769
Epoch 26/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.3740 - accuracy: 0.8685 - val_loss: 1.2622 - val_accuracy: 0.6748

Running privacy report for epoch: 26

Epoch 27/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.3648 - accuracy: 0.8709 - val_loss: 1.2614 - val_accuracy: 0.6813
Epoch 28/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.3479 - accuracy: 0.8758 - val_loss: 1.3504 - val_accuracy: 0.6716

Running privacy report for epoch: 28

Epoch 29/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.3306 - accuracy: 0.8835 - val_loss: 1.3365 - val_accuracy: 0.6812
Epoch 30/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.3234 - accuracy: 0.8850 - val_loss: 1.3883 - val_accuracy: 0.6766

Running privacy report for epoch: 30

Epoch 31/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.3125 - accuracy: 0.8887 - val_loss: 1.3960 - val_accuracy: 0.6747
Epoch 32/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.2956 - accuracy: 0.8938 - val_loss: 1.4326 - val_accuracy: 0.6780

Running privacy report for epoch: 32

Epoch 33/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.2903 - accuracy: 0.8955 - val_loss: 1.5069 - val_accuracy: 0.6717
Epoch 34/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.2752 - accuracy: 0.9016 - val_loss: 1.5441 - val_accuracy: 0.6691

Running privacy report for epoch: 34

Epoch 35/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.2676 - accuracy: 0.9056 - val_loss: 1.5704 - val_accuracy: 0.6633
Epoch 36/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.2633 - accuracy: 0.9053 - val_loss: 1.6393 - val_accuracy: 0.6768

Running privacy report for epoch: 36

Epoch 37/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.2472 - accuracy: 0.9113 - val_loss: 1.6478 - val_accuracy: 0.6690
Epoch 38/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.2413 - accuracy: 0.9148 - val_loss: 1.7070 - val_accuracy: 0.6683

Running privacy report for epoch: 38

Epoch 39/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.2262 - accuracy: 0.9191 - val_loss: 1.7845 - val_accuracy: 0.6599
Epoch 40/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.2297 - accuracy: 0.9171 - val_loss: 1.8155 - val_accuracy: 0.6683

Running privacy report for epoch: 40

Epoch 41/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.2231 - accuracy: 0.9202 - val_loss: 1.8371 - val_accuracy: 0.6636
Epoch 42/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.2051 - accuracy: 0.9264 - val_loss: 1.9123 - val_accuracy: 0.6677

Running privacy report for epoch: 42

Epoch 43/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.2064 - accuracy: 0.9257 - val_loss: 1.9646 - val_accuracy: 0.6618
Epoch 44/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.1952 - accuracy: 0.9315 - val_loss: 1.9862 - val_accuracy: 0.6606

Running privacy report for epoch: 44

Epoch 45/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.1845 - accuracy: 0.9340 - val_loss: 2.0513 - val_accuracy: 0.6645
Epoch 46/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.1859 - accuracy: 0.9334 - val_loss: 2.0950 - val_accuracy: 0.6629

Running privacy report for epoch: 46

Epoch 47/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.1746 - accuracy: 0.9381 - val_loss: 2.0852 - val_accuracy: 0.6640
Epoch 48/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.1751 - accuracy: 0.9369 - val_loss: 2.1738 - val_accuracy: 0.6668

Running privacy report for epoch: 48

Epoch 49/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.1725 - accuracy: 0.9384 - val_loss: 2.2343 - val_accuracy: 0.6647
Epoch 50/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.1681 - accuracy: 0.9401 - val_loss: 2.3633 - val_accuracy: 0.6553

Running privacy report for epoch: 50
callback = PrivacyMetrics(epochs_per_report, "3 Layers")
history = model_3layers.fit(
      x_train,
      y_train,
      batch_size=batch_size,
      epochs=total_epochs,
      validation_data=(x_test, y_test),
      callbacks=[callback],
      shuffle=True)

all_reports.extend(callback.attack_results)
Epoch 1/50
1000/1000 [==============================] - 4s 3ms/step - loss: 1.6652 - accuracy: 0.3893 - val_loss: 1.4180 - val_accuracy: 0.4964
Epoch 2/50
1000/1000 [==============================] - 3s 3ms/step - loss: 1.3396 - accuracy: 0.5194 - val_loss: 1.2793 - val_accuracy: 0.5410

Running privacy report for epoch: 2

Epoch 3/50
1000/1000 [==============================] - 3s 3ms/step - loss: 1.2242 - accuracy: 0.5656 - val_loss: 1.2294 - val_accuracy: 0.5666
Epoch 4/50
1000/1000 [==============================] - 3s 3ms/step - loss: 1.1552 - accuracy: 0.5891 - val_loss: 1.1869 - val_accuracy: 0.5799

Running privacy report for epoch: 4

Epoch 5/50
1000/1000 [==============================] - 3s 3ms/step - loss: 1.0892 - accuracy: 0.6134 - val_loss: 1.1003 - val_accuracy: 0.6168
Epoch 6/50
1000/1000 [==============================] - 3s 3ms/step - loss: 1.0405 - accuracy: 0.6338 - val_loss: 1.1087 - val_accuracy: 0.6161

Running privacy report for epoch: 6

Epoch 7/50
1000/1000 [==============================] - 3s 3ms/step - loss: 1.0009 - accuracy: 0.6478 - val_loss: 1.0654 - val_accuracy: 0.6275
Epoch 8/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.9625 - accuracy: 0.6628 - val_loss: 1.0217 - val_accuracy: 0.6462

Running privacy report for epoch: 8

Epoch 9/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.9328 - accuracy: 0.6723 - val_loss: 0.9918 - val_accuracy: 0.6552
Epoch 10/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.9042 - accuracy: 0.6820 - val_loss: 0.9884 - val_accuracy: 0.6578

Running privacy report for epoch: 10

Epoch 11/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.8763 - accuracy: 0.6921 - val_loss: 0.9602 - val_accuracy: 0.6680
Epoch 12/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.8570 - accuracy: 0.6990 - val_loss: 0.9734 - val_accuracy: 0.6588

Running privacy report for epoch: 12

Epoch 13/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.8317 - accuracy: 0.7072 - val_loss: 0.9562 - val_accuracy: 0.6648
Epoch 14/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.8205 - accuracy: 0.7121 - val_loss: 0.9584 - val_accuracy: 0.6679

Running privacy report for epoch: 14

Epoch 15/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.7981 - accuracy: 0.7189 - val_loss: 0.9686 - val_accuracy: 0.6685
Epoch 16/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.7805 - accuracy: 0.7231 - val_loss: 0.9379 - val_accuracy: 0.6710

Running privacy report for epoch: 16

Epoch 17/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.7680 - accuracy: 0.7296 - val_loss: 0.9345 - val_accuracy: 0.6785
Epoch 18/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.7518 - accuracy: 0.7350 - val_loss: 0.9055 - val_accuracy: 0.6828

Running privacy report for epoch: 18

Epoch 19/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.7432 - accuracy: 0.7371 - val_loss: 0.8970 - val_accuracy: 0.6903
Epoch 20/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.7277 - accuracy: 0.7432 - val_loss: 0.9244 - val_accuracy: 0.6851

Running privacy report for epoch: 20

Epoch 21/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.7202 - accuracy: 0.7438 - val_loss: 0.9329 - val_accuracy: 0.6796
Epoch 22/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.7083 - accuracy: 0.7497 - val_loss: 0.9210 - val_accuracy: 0.6875

Running privacy report for epoch: 22

Epoch 23/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.6944 - accuracy: 0.7546 - val_loss: 0.9225 - val_accuracy: 0.6827
Epoch 24/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.6911 - accuracy: 0.7564 - val_loss: 0.9318 - val_accuracy: 0.6872

Running privacy report for epoch: 24

Epoch 25/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.6794 - accuracy: 0.7607 - val_loss: 0.9418 - val_accuracy: 0.6821
Epoch 26/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.6733 - accuracy: 0.7625 - val_loss: 0.9584 - val_accuracy: 0.6809

Running privacy report for epoch: 26

Epoch 27/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.6594 - accuracy: 0.7654 - val_loss: 0.9338 - val_accuracy: 0.6887
Epoch 28/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.6513 - accuracy: 0.7675 - val_loss: 0.9485 - val_accuracy: 0.6884

Running privacy report for epoch: 28

Epoch 29/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.6434 - accuracy: 0.7723 - val_loss: 0.9487 - val_accuracy: 0.6813
Epoch 30/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.6324 - accuracy: 0.7751 - val_loss: 0.9274 - val_accuracy: 0.6947

Running privacy report for epoch: 30

Epoch 31/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.6261 - accuracy: 0.7792 - val_loss: 0.9306 - val_accuracy: 0.6869
Epoch 32/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.6206 - accuracy: 0.7784 - val_loss: 0.9274 - val_accuracy: 0.6977

Running privacy report for epoch: 32

Epoch 33/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.6177 - accuracy: 0.7803 - val_loss: 0.9473 - val_accuracy: 0.6927
Epoch 34/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.6042 - accuracy: 0.7870 - val_loss: 0.9426 - val_accuracy: 0.6934

Running privacy report for epoch: 34

Epoch 35/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.5945 - accuracy: 0.7879 - val_loss: 0.9593 - val_accuracy: 0.6891
Epoch 36/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.5912 - accuracy: 0.7895 - val_loss: 0.9772 - val_accuracy: 0.6878

Running privacy report for epoch: 36

Epoch 37/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.5847 - accuracy: 0.7910 - val_loss: 0.9324 - val_accuracy: 0.6972
Epoch 38/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.5748 - accuracy: 0.7930 - val_loss: 0.9729 - val_accuracy: 0.6959

Running privacy report for epoch: 38

Epoch 39/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.5729 - accuracy: 0.7960 - val_loss: 0.9839 - val_accuracy: 0.6933
Epoch 40/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.5703 - accuracy: 0.7962 - val_loss: 0.9680 - val_accuracy: 0.6949

Running privacy report for epoch: 40

Epoch 41/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.5615 - accuracy: 0.8005 - val_loss: 1.0089 - val_accuracy: 0.6906
Epoch 42/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.5542 - accuracy: 0.8015 - val_loss: 0.9932 - val_accuracy: 0.6903

Running privacy report for epoch: 42

Epoch 43/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.5499 - accuracy: 0.8026 - val_loss: 0.9847 - val_accuracy: 0.6940
Epoch 44/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.5408 - accuracy: 0.8067 - val_loss: 1.0146 - val_accuracy: 0.6883

Running privacy report for epoch: 44

Epoch 45/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.5376 - accuracy: 0.8082 - val_loss: 0.9935 - val_accuracy: 0.6926
Epoch 46/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.5327 - accuracy: 0.8072 - val_loss: 1.0227 - val_accuracy: 0.6904

Running privacy report for epoch: 46

Epoch 47/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.5304 - accuracy: 0.8096 - val_loss: 1.0110 - val_accuracy: 0.6966
Epoch 48/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.5205 - accuracy: 0.8135 - val_loss: 1.0087 - val_accuracy: 0.6899

Running privacy report for epoch: 48

Epoch 49/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.5235 - accuracy: 0.8107 - val_loss: 0.9936 - val_accuracy: 0.6988
Epoch 50/50
1000/1000 [==============================] - 3s 3ms/step - loss: 0.5114 - accuracy: 0.8178 - val_loss: 1.0252 - val_accuracy: 0.6953

Running privacy report for epoch: 50

Epoch Plots

You can visualize how privacy risks happen as you train models by probing the model periodically (e.g. every 5 epochs), you can pick the point in time with the best performance / privacy trade-off.

Use the TF Privacy Membership Inference Attack module to generate AttackResults. These AttackResults get combined into an AttackResultsCollection. The TF Privacy Report is designed to analyze the provided AttackResultsCollection.

results = AttackResultsCollection(all_reports)
privacy_metrics = (PrivacyMetric.AUC, PrivacyMetric.ATTACKER_ADVANTAGE)
epoch_plot = privacy_report.plot_by_epochs(
    results, privacy_metrics=privacy_metrics)

png

See that as a rule, privacy vulnerability tends to increase as the number of epochs goes up. This is true across model variants as well as different attacker types.

Two layer models (with fewer convolutional layers) are generally more vulnerable than their three layer model counterparts.

Now let's see how model performance changes with respect to privacy risk.

Privacy vs Utility

privacy_metrics = (PrivacyMetric.AUC, PrivacyMetric.ATTACKER_ADVANTAGE)
utility_privacy_plot = privacy_report.plot_privacy_vs_accuracy(
    results, privacy_metrics=privacy_metrics)

for axis in utility_privacy_plot.axes:
  axis.set_xlabel('Validation accuracy')

png

Three layer models (perhaps due to too many parameters) only achieve a train accuracy of 0.85. The two layer models achieve roughly equal performance for that level of privacy risk but they continue to get better accuracy.

You can also see how the line for two layer models gets steeper. This means that additional marginal gains in train accuracy come at an expense of vast privacy vulnerabilities.

This is the end of the tutorial. Feel free to analyze your own results.