Assess privacy risks with the TensorFlow Privacy Report

View on TensorFlow.org Run in Google Colab View source on GitHub Download notebook

Overview

In this codelab you'll train a simple image classification model on the CIFAR10 dataset, and then use the "membership inference attack" against this model to assess if the attacker is able to "guess" whether a particular sample was present in the training set. You will use the TF Privacy Report to visualize results from multiple models and model checkpoints.

Setup

import numpy as np
from typing import Tuple
from scipy import special
from sklearn import metrics

import tensorflow as tf

import tensorflow_datasets as tfds

# Set verbosity.
tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)
from sklearn.exceptions import ConvergenceWarning

import warnings
warnings.simplefilter(action="ignore", category=ConvergenceWarning)
warnings.simplefilter(action="ignore", category=FutureWarning)

Install TensorFlow Privacy.

pip install tensorflow_privacy
from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack import membership_inference_attack as mia
from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack.data_structures import AttackInputData
from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack.data_structures import AttackResultsCollection
from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack.data_structures import AttackType
from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack.data_structures import PrivacyMetric
from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack.data_structures import PrivacyReportMetadata
from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack.data_structures import SlicingSpec
from tensorflow_privacy.privacy.privacy_tests.membership_inference_attack import privacy_report
import tensorflow_privacy

Train two models, with privacy metrics

This section trains a pair of keras.Model classifiers on the CIFAR-10 dataset. During the training process it collects privacy metrics, that will be used to generate reports in the bext section.

The first step is to define some hyperparameters:

dataset = 'cifar10'
num_classes = 10
activation = 'relu'
num_conv = 3

batch_size=50
epochs_per_report = 2
total_epochs = 50

lr = 0.001

Next, load the dataset. There's nothing privacy-specific in this code.

Loading the dataset.

Next define a function to build the models.

Build two three-layer CNN models using that function.

Configure the first to use a basic SGD optimizer, an the second to use a differentially private optimizer (tf_privacy.DPKerasAdamOptimizer), so you can compare the results.

model_2layers = small_cnn(
    input_shape, num_classes, num_conv=2, activation=activation)
model_3layers = small_cnn(
    input_shape, num_classes, num_conv=3, activation=activation)

Define a callback to collect privacy metrics

Next define a keras.callbacks.Callback to periorically run some privacy attacks against the model, and log the results.

The keras fit method will call the on_epoch_end method after each training epoch. The n argument is the (0-based) epoch number.

You could implement this procedure by writing a loop that repeatedly calls Model.fit(..., epochs=epochs_per_report) and runs the attack code. The callback is used here just because it gives a clear separation between the training logic, and the privacy evaluation logic.

class PrivacyMetrics(tf.keras.callbacks.Callback):
  def __init__(self, epochs_per_report, model_name):
    self.epochs_per_report = epochs_per_report
    self.model_name = model_name
    self.attack_results = []

  def on_epoch_end(self, epoch, logs=None):
    epoch = epoch+1

    if epoch % self.epochs_per_report != 0:
      return

    print(f'\nRunning privacy report for epoch: {epoch}\n')

    logits_train = self.model.predict(x_train, batch_size=batch_size)
    logits_test = self.model.predict(x_test, batch_size=batch_size)

    prob_train = special.softmax(logits_train, axis=1)
    prob_test = special.softmax(logits_test, axis=1)

    # Add metadata to generate a privacy report.
    privacy_report_metadata = PrivacyReportMetadata(
        # Show the validation accuracy on the plot
        # It's what you send to train_accuracy that gets plotted.
        accuracy_train=logs['val_accuracy'], 
        accuracy_test=logs['val_accuracy'],
        epoch_num=epoch,
        model_variant_label=self.model_name)

    attack_results = mia.run_attacks(
        AttackInputData(
            labels_train=y_train_indices[:, 0],
            labels_test=y_test_indices[:, 0],
            probs_train=prob_train,
            probs_test=prob_test),
        SlicingSpec(entire_dataset=True, by_class=True),
        attack_types=(AttackType.THRESHOLD_ATTACK,
                      AttackType.LOGISTIC_REGRESSION),
        privacy_report_metadata=privacy_report_metadata)

    self.attack_results.append(attack_results)

Train the models

The next code block trains the two models. The all_reports list is used to collect all the results from all the models' training runs. The individual reports are tagged witht the model_name, so there's no confusion about which model generated which report.

all_reports = []
callback = PrivacyMetrics(epochs_per_report, "2 Layers")
history = model_2layers.fit(
      x_train,
      y_train,
      batch_size=batch_size,
      epochs=total_epochs,
      validation_data=(x_test, y_test),
      callbacks=[callback],
      shuffle=True)

all_reports.extend(callback.attack_results)
Epoch 1/50
1000/1000 [==============================] - 6s 4ms/step - loss: 1.5138 - accuracy: 0.4539 - val_loss: 1.2587 - val_accuracy: 0.5544
Epoch 2/50
 998/1000 [============================>.] - ETA: 0s - loss: 1.1648 - accuracy: 0.5928
Running privacy report for epoch: 2

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 16s 16ms/step - loss: 1.1650 - accuracy: 0.5927 - val_loss: 1.1076 - val_accuracy: 0.6116
Epoch 3/50
1000/1000 [==============================] - 4s 4ms/step - loss: 1.0424 - accuracy: 0.6387 - val_loss: 1.0152 - val_accuracy: 0.6473
Epoch 4/50
 984/1000 [============================>.] - ETA: 0s - loss: 0.9675 - accuracy: 0.6627
Running privacy report for epoch: 4

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 16s 16ms/step - loss: 0.9657 - accuracy: 0.6632 - val_loss: 0.9973 - val_accuracy: 0.6532
Epoch 5/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.9145 - accuracy: 0.6807 - val_loss: 0.9619 - val_accuracy: 0.6680
Epoch 6/50
 988/1000 [============================>.] - ETA: 0s - loss: 0.8627 - accuracy: 0.6999
Running privacy report for epoch: 6

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 16s 16ms/step - loss: 0.8627 - accuracy: 0.6997 - val_loss: 0.9801 - val_accuracy: 0.6601
Epoch 7/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.8267 - accuracy: 0.7117 - val_loss: 0.9134 - val_accuracy: 0.6866
Epoch 8/50
 988/1000 [============================>.] - ETA: 0s - loss: 0.7903 - accuracy: 0.7249
Running privacy report for epoch: 8

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 16s 16ms/step - loss: 0.7914 - accuracy: 0.7245 - val_loss: 0.9257 - val_accuracy: 0.6805
Epoch 9/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.7606 - accuracy: 0.7335 - val_loss: 0.9131 - val_accuracy: 0.6857
Epoch 10/50
 995/1000 [============================>.] - ETA: 0s - loss: 0.7314 - accuracy: 0.7443
Running privacy report for epoch: 10

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 16s 16ms/step - loss: 0.7314 - accuracy: 0.7440 - val_loss: 0.8934 - val_accuracy: 0.6979
Epoch 11/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.7042 - accuracy: 0.7545 - val_loss: 0.9100 - val_accuracy: 0.6931
Epoch 12/50
 989/1000 [============================>.] - ETA: 0s - loss: 0.6817 - accuracy: 0.7603
Running privacy report for epoch: 12

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 16s 16ms/step - loss: 0.6818 - accuracy: 0.7603 - val_loss: 0.9105 - val_accuracy: 0.6953
Epoch 13/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.6551 - accuracy: 0.7716 - val_loss: 0.9342 - val_accuracy: 0.6915
Epoch 14/50
 998/1000 [============================>.] - ETA: 0s - loss: 0.6333 - accuracy: 0.7769
Running privacy report for epoch: 14

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 16s 16ms/step - loss: 0.6333 - accuracy: 0.7769 - val_loss: 0.9402 - val_accuracy: 0.6960
Epoch 15/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.6088 - accuracy: 0.7887 - val_loss: 0.9429 - val_accuracy: 0.6954
Epoch 16/50
 986/1000 [============================>.] - ETA: 0s - loss: 0.5870 - accuracy: 0.7945
Running privacy report for epoch: 16

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 16s 16ms/step - loss: 0.5874 - accuracy: 0.7945 - val_loss: 0.9352 - val_accuracy: 0.6963
Epoch 17/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.5707 - accuracy: 0.7991 - val_loss: 0.9745 - val_accuracy: 0.6877
Epoch 18/50
 993/1000 [============================>.] - ETA: 0s - loss: 0.5470 - accuracy: 0.8076
Running privacy report for epoch: 18

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 16s 16ms/step - loss: 0.5471 - accuracy: 0.8076 - val_loss: 0.9592 - val_accuracy: 0.6983
Epoch 19/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.5321 - accuracy: 0.8131 - val_loss: 0.9993 - val_accuracy: 0.6897
Epoch 20/50
 995/1000 [============================>.] - ETA: 0s - loss: 0.5105 - accuracy: 0.8183
Running privacy report for epoch: 20

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 16s 16ms/step - loss: 0.5099 - accuracy: 0.8186 - val_loss: 0.9785 - val_accuracy: 0.7012
Epoch 21/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.4901 - accuracy: 0.8286 - val_loss: 1.0197 - val_accuracy: 0.6932
Epoch 22/50
 990/1000 [============================>.] - ETA: 0s - loss: 0.4784 - accuracy: 0.8293
Running privacy report for epoch: 22

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.4792 - accuracy: 0.8290 - val_loss: 1.0571 - val_accuracy: 0.6858
Epoch 23/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.4615 - accuracy: 0.8382 - val_loss: 1.0692 - val_accuracy: 0.6877
Epoch 24/50
 984/1000 [============================>.] - ETA: 0s - loss: 0.4482 - accuracy: 0.8404
Running privacy report for epoch: 24

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.4498 - accuracy: 0.8399 - val_loss: 1.1347 - val_accuracy: 0.6754
Epoch 25/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.4330 - accuracy: 0.8461 - val_loss: 1.1000 - val_accuracy: 0.6896
Epoch 26/50
 998/1000 [============================>.] - ETA: 0s - loss: 0.4197 - accuracy: 0.8516
Running privacy report for epoch: 26

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.4198 - accuracy: 0.8516 - val_loss: 1.1110 - val_accuracy: 0.6901
Epoch 27/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.4075 - accuracy: 0.8560 - val_loss: 1.1396 - val_accuracy: 0.6931
Epoch 28/50
 988/1000 [============================>.] - ETA: 0s - loss: 0.3949 - accuracy: 0.8602
Running privacy report for epoch: 28

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.3954 - accuracy: 0.8600 - val_loss: 1.1926 - val_accuracy: 0.6801
Epoch 29/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.3829 - accuracy: 0.8625 - val_loss: 1.2057 - val_accuracy: 0.6859
Epoch 30/50
 987/1000 [============================>.] - ETA: 0s - loss: 0.3742 - accuracy: 0.8654
Running privacy report for epoch: 30

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.3739 - accuracy: 0.8654 - val_loss: 1.2535 - val_accuracy: 0.6812
Epoch 31/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.3597 - accuracy: 0.8720 - val_loss: 1.2707 - val_accuracy: 0.6793
Epoch 32/50
 986/1000 [============================>.] - ETA: 0s - loss: 0.3450 - accuracy: 0.8769
Running privacy report for epoch: 32

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.3449 - accuracy: 0.8769 - val_loss: 1.2817 - val_accuracy: 0.6841
Epoch 33/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.3351 - accuracy: 0.8805 - val_loss: 1.3555 - val_accuracy: 0.6814
Epoch 34/50
 986/1000 [============================>.] - ETA: 0s - loss: 0.3200 - accuracy: 0.8879
Running privacy report for epoch: 34

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.3212 - accuracy: 0.8875 - val_loss: 1.3778 - val_accuracy: 0.6783
Epoch 35/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.3185 - accuracy: 0.8851 - val_loss: 1.4277 - val_accuracy: 0.6744
Epoch 36/50
 992/1000 [============================>.] - ETA: 0s - loss: 0.3089 - accuracy: 0.8894
Running privacy report for epoch: 36

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.3093 - accuracy: 0.8893 - val_loss: 1.4060 - val_accuracy: 0.6743
Epoch 37/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.2970 - accuracy: 0.8940 - val_loss: 1.4602 - val_accuracy: 0.6751
Epoch 38/50
 991/1000 [============================>.] - ETA: 0s - loss: 0.2873 - accuracy: 0.8972
Running privacy report for epoch: 38

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.2876 - accuracy: 0.8970 - val_loss: 1.4984 - val_accuracy: 0.6688
Epoch 39/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.2760 - accuracy: 0.9012 - val_loss: 1.5528 - val_accuracy: 0.6730
Epoch 40/50
 991/1000 [============================>.] - ETA: 0s - loss: 0.2747 - accuracy: 0.9011
Running privacy report for epoch: 40

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 16s 16ms/step - loss: 0.2751 - accuracy: 0.9010 - val_loss: 1.5654 - val_accuracy: 0.6708
Epoch 41/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.2603 - accuracy: 0.9064 - val_loss: 1.6609 - val_accuracy: 0.6671
Epoch 42/50
 986/1000 [============================>.] - ETA: 0s - loss: 0.2521 - accuracy: 0.9093
Running privacy report for epoch: 42

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.2526 - accuracy: 0.9090 - val_loss: 1.7333 - val_accuracy: 0.6618
Epoch 43/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.2481 - accuracy: 0.9097 - val_loss: 1.7160 - val_accuracy: 0.6635
Epoch 44/50
 987/1000 [============================>.] - ETA: 0s - loss: 0.2391 - accuracy: 0.9130
Running privacy report for epoch: 44

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.2397 - accuracy: 0.9126 - val_loss: 1.7751 - val_accuracy: 0.6703
Epoch 45/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.2360 - accuracy: 0.9147 - val_loss: 1.8201 - val_accuracy: 0.6672
Epoch 46/50
 997/1000 [============================>.] - ETA: 0s - loss: 0.2229 - accuracy: 0.9195
Running privacy report for epoch: 46

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 16s 16ms/step - loss: 0.2231 - accuracy: 0.9195 - val_loss: 1.8708 - val_accuracy: 0.6620
Epoch 47/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.2235 - accuracy: 0.9190 - val_loss: 1.9290 - val_accuracy: 0.6532
Epoch 48/50
 994/1000 [============================>.] - ETA: 0s - loss: 0.2207 - accuracy: 0.9192
Running privacy report for epoch: 48

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.2208 - accuracy: 0.9191 - val_loss: 1.9354 - val_accuracy: 0.6690
Epoch 49/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.2144 - accuracy: 0.9226 - val_loss: 2.0203 - val_accuracy: 0.6620
Epoch 50/50
 985/1000 [============================>.] - ETA: 0s - loss: 0.2031 - accuracy: 0.9260
Running privacy report for epoch: 50

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 16s 16ms/step - loss: 0.2036 - accuracy: 0.9258 - val_loss: 1.9738 - val_accuracy: 0.6661
callback = PrivacyMetrics(epochs_per_report, "3 Layers")
history = model_3layers.fit(
      x_train,
      y_train,
      batch_size=batch_size,
      epochs=total_epochs,
      validation_data=(x_test, y_test),
      callbacks=[callback],
      shuffle=True)

all_reports.extend(callback.attack_results)
Epoch 1/50
1000/1000 [==============================] - 5s 4ms/step - loss: 1.6620 - accuracy: 0.3867 - val_loss: 1.4188 - val_accuracy: 0.4801
Epoch 2/50
 990/1000 [============================>.] - ETA: 0s - loss: 1.3098 - accuracy: 0.5288
Running privacy report for epoch: 2

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 1.3100 - accuracy: 0.5289 - val_loss: 1.2532 - val_accuracy: 0.5464
Epoch 3/50
1000/1000 [==============================] - 4s 4ms/step - loss: 1.1888 - accuracy: 0.5776 - val_loss: 1.1515 - val_accuracy: 0.5944
Epoch 4/50
 997/1000 [============================>.] - ETA: 0s - loss: 1.1116 - accuracy: 0.6078
Running privacy report for epoch: 4

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 16s 16ms/step - loss: 1.1115 - accuracy: 0.6079 - val_loss: 1.0947 - val_accuracy: 0.6086
Epoch 5/50
1000/1000 [==============================] - 4s 4ms/step - loss: 1.0571 - accuracy: 0.6284 - val_loss: 1.0415 - val_accuracy: 0.6340
Epoch 6/50
 993/1000 [============================>.] - ETA: 0s - loss: 1.0058 - accuracy: 0.6471
Running privacy report for epoch: 6

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 16s 16ms/step - loss: 1.0051 - accuracy: 0.6476 - val_loss: 1.0299 - val_accuracy: 0.6374
Epoch 7/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.9626 - accuracy: 0.6633 - val_loss: 1.0105 - val_accuracy: 0.6484
Epoch 8/50
 992/1000 [============================>.] - ETA: 0s - loss: 0.9317 - accuracy: 0.6748
Running privacy report for epoch: 8

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 16s 16ms/step - loss: 0.9313 - accuracy: 0.6750 - val_loss: 1.0389 - val_accuracy: 0.6417
Epoch 9/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.9005 - accuracy: 0.6863 - val_loss: 0.9568 - val_accuracy: 0.6650
Epoch 10/50
 997/1000 [============================>.] - ETA: 0s - loss: 0.8731 - accuracy: 0.6940
Running privacy report for epoch: 10

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 16s 16ms/step - loss: 0.8728 - accuracy: 0.6942 - val_loss: 0.9554 - val_accuracy: 0.6675
Epoch 11/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.8514 - accuracy: 0.7031 - val_loss: 0.9868 - val_accuracy: 0.6585
Epoch 12/50
 998/1000 [============================>.] - ETA: 0s - loss: 0.8329 - accuracy: 0.7101
Running privacy report for epoch: 12

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 16s 16ms/step - loss: 0.8325 - accuracy: 0.7102 - val_loss: 0.9283 - val_accuracy: 0.6771
Epoch 13/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.8056 - accuracy: 0.7185 - val_loss: 0.9553 - val_accuracy: 0.6658
Epoch 14/50
1000/1000 [==============================] - ETA: 0s - loss: 0.7878 - accuracy: 0.7222
Running privacy report for epoch: 14

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 16s 16ms/step - loss: 0.7878 - accuracy: 0.7222 - val_loss: 0.9054 - val_accuracy: 0.6869
Epoch 15/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.7729 - accuracy: 0.7295 - val_loss: 0.9346 - val_accuracy: 0.6822
Epoch 16/50
 986/1000 [============================>.] - ETA: 0s - loss: 0.7502 - accuracy: 0.7382
Running privacy report for epoch: 16

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.7510 - accuracy: 0.7376 - val_loss: 0.9321 - val_accuracy: 0.6857
Epoch 17/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.7349 - accuracy: 0.7419 - val_loss: 0.9053 - val_accuracy: 0.6904
Epoch 18/50
 998/1000 [============================>.] - ETA: 0s - loss: 0.7258 - accuracy: 0.7464
Running privacy report for epoch: 18

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 16s 16ms/step - loss: 0.7257 - accuracy: 0.7464 - val_loss: 0.8969 - val_accuracy: 0.6976
Epoch 19/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.7112 - accuracy: 0.7499 - val_loss: 0.9086 - val_accuracy: 0.6908
Epoch 20/50
 999/1000 [============================>.] - ETA: 0s - loss: 0.6967 - accuracy: 0.7547
Running privacy report for epoch: 20

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.6966 - accuracy: 0.7547 - val_loss: 0.9142 - val_accuracy: 0.6849
Epoch 21/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.6878 - accuracy: 0.7585 - val_loss: 0.9392 - val_accuracy: 0.6903
Epoch 22/50
 996/1000 [============================>.] - ETA: 0s - loss: 0.6740 - accuracy: 0.7625
Running privacy report for epoch: 22

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.6741 - accuracy: 0.7624 - val_loss: 0.9317 - val_accuracy: 0.6881
Epoch 23/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.6606 - accuracy: 0.7674 - val_loss: 0.9017 - val_accuracy: 0.6959
Epoch 24/50
 989/1000 [============================>.] - ETA: 0s - loss: 0.6578 - accuracy: 0.7671
Running privacy report for epoch: 24

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.6578 - accuracy: 0.7669 - val_loss: 0.9128 - val_accuracy: 0.6963
Epoch 25/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.6416 - accuracy: 0.7760 - val_loss: 0.9372 - val_accuracy: 0.6846
Epoch 26/50
 986/1000 [============================>.] - ETA: 0s - loss: 0.6367 - accuracy: 0.7761
Running privacy report for epoch: 26

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.6371 - accuracy: 0.7761 - val_loss: 0.8943 - val_accuracy: 0.6985
Epoch 27/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.6282 - accuracy: 0.7787 - val_loss: 0.8978 - val_accuracy: 0.7028
Epoch 28/50
 995/1000 [============================>.] - ETA: 0s - loss: 0.6143 - accuracy: 0.7834
Running privacy report for epoch: 28

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.6145 - accuracy: 0.7833 - val_loss: 0.8964 - val_accuracy: 0.7010
Epoch 29/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.6097 - accuracy: 0.7858 - val_loss: 0.9374 - val_accuracy: 0.6976
Epoch 30/50
 995/1000 [============================>.] - ETA: 0s - loss: 0.6051 - accuracy: 0.7859
Running privacy report for epoch: 30

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.6052 - accuracy: 0.7859 - val_loss: 0.9184 - val_accuracy: 0.6957
Epoch 31/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.5934 - accuracy: 0.7909 - val_loss: 0.9247 - val_accuracy: 0.7004
Epoch 32/50
 993/1000 [============================>.] - ETA: 0s - loss: 0.5886 - accuracy: 0.7927
Running privacy report for epoch: 32

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.5890 - accuracy: 0.7926 - val_loss: 0.9255 - val_accuracy: 0.7016
Epoch 33/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.5833 - accuracy: 0.7934 - val_loss: 0.9430 - val_accuracy: 0.6957
Epoch 34/50
 999/1000 [============================>.] - ETA: 0s - loss: 0.5792 - accuracy: 0.7956
Running privacy report for epoch: 34

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.5792 - accuracy: 0.7956 - val_loss: 0.9923 - val_accuracy: 0.6866
Epoch 35/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.5667 - accuracy: 0.7987 - val_loss: 0.9665 - val_accuracy: 0.6963
Epoch 36/50
 991/1000 [============================>.] - ETA: 0s - loss: 0.5657 - accuracy: 0.7981
Running privacy report for epoch: 36

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.5659 - accuracy: 0.7981 - val_loss: 0.9579 - val_accuracy: 0.6952
Epoch 37/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.5536 - accuracy: 0.8032 - val_loss: 0.9302 - val_accuracy: 0.7012
Epoch 38/50
 991/1000 [============================>.] - ETA: 0s - loss: 0.5531 - accuracy: 0.8036
Running privacy report for epoch: 38

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.5539 - accuracy: 0.8035 - val_loss: 0.9817 - val_accuracy: 0.6881
Epoch 39/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.5455 - accuracy: 0.8063 - val_loss: 0.9439 - val_accuracy: 0.7014
Epoch 40/50
 992/1000 [============================>.] - ETA: 0s - loss: 0.5427 - accuracy: 0.8070
Running privacy report for epoch: 40

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.5428 - accuracy: 0.8070 - val_loss: 0.9475 - val_accuracy: 0.6986
Epoch 41/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.5284 - accuracy: 0.8127 - val_loss: 1.0001 - val_accuracy: 0.6939
Epoch 42/50
 987/1000 [============================>.] - ETA: 0s - loss: 0.5284 - accuracy: 0.8105
Running privacy report for epoch: 42

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.5291 - accuracy: 0.8103 - val_loss: 0.9639 - val_accuracy: 0.6983
Epoch 43/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.5231 - accuracy: 0.8150 - val_loss: 0.9585 - val_accuracy: 0.7023
Epoch 44/50
1000/1000 [==============================] - ETA: 0s - loss: 0.5226 - accuracy: 0.8129
Running privacy report for epoch: 44

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 18s 18ms/step - loss: 0.5226 - accuracy: 0.8129 - val_loss: 1.0242 - val_accuracy: 0.6827
Epoch 45/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.5147 - accuracy: 0.8172 - val_loss: 1.0032 - val_accuracy: 0.6923
Epoch 46/50
 988/1000 [============================>.] - ETA: 0s - loss: 0.5079 - accuracy: 0.8208
Running privacy report for epoch: 46

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.5084 - accuracy: 0.8205 - val_loss: 1.0149 - val_accuracy: 0.6904
Epoch 47/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.4997 - accuracy: 0.8203 - val_loss: 1.0016 - val_accuracy: 0.6940
Epoch 48/50
 987/1000 [============================>.] - ETA: 0s - loss: 0.5002 - accuracy: 0.8241
Running privacy report for epoch: 48

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.5001 - accuracy: 0.8240 - val_loss: 1.0309 - val_accuracy: 0.6945
Epoch 49/50
1000/1000 [==============================] - 4s 4ms/step - loss: 0.4978 - accuracy: 0.8235 - val_loss: 1.0311 - val_accuracy: 0.6908
Epoch 50/50
 998/1000 [============================>.] - ETA: 0s - loss: 0.4892 - accuracy: 0.8265
Running privacy report for epoch: 50

1000/1000 [==============================] - 2s 2ms/step
200/200 [==============================] - 0s 2ms/step
1000/1000 [==============================] - 17s 17ms/step - loss: 0.4893 - accuracy: 0.8265 - val_loss: 1.0204 - val_accuracy: 0.6951

Epoch Plots

You can visualize how privacy risks happen as you train models by probing the model periodically (e.g. every 5 epochs), you can pick the point in time with the best performance / privacy trade-off.

Use the TF Privacy Membership Inference Attack module to generate AttackResults. These AttackResults get combined into an AttackResultsCollection. The TF Privacy Report is designed to analyze the provided AttackResultsCollection.

results = AttackResultsCollection(all_reports)
privacy_metrics = (PrivacyMetric.AUC, PrivacyMetric.ATTACKER_ADVANTAGE)
epoch_plot = privacy_report.plot_by_epochs(
    results, privacy_metrics=privacy_metrics)

png

See that as a rule, privacy vulnerability tends to increase as the number of epochs goes up. This is true across model variants as well as different attacker types.

Two layer models (with fewer convolutional layers) are generally more vulnerable than their three layer model counterparts.

Now let's see how model performance changes with respect to privacy risk.

Privacy vs Utility

privacy_metrics = (PrivacyMetric.AUC, PrivacyMetric.ATTACKER_ADVANTAGE)
utility_privacy_plot = privacy_report.plot_privacy_vs_accuracy(
    results, privacy_metrics=privacy_metrics)

for axis in utility_privacy_plot.axes:
  axis.set_xlabel('Validation accuracy')

png

Three layer models (perhaps due to too many parameters) only achieve a train accuracy of 0.85. The two layer models achieve roughly equal performance for that level of privacy risk but they continue to get better accuracy.

You can also see how the line for two layer models gets steeper. This means that additional marginal gains in train accuracy come at an expense of vast privacy vulnerabilities.

This is the end of the tutorial. Feel free to analyze your own results.