TFF simulations on GCP

This tutorial will describe how to run TFF simulations on GCP.

Run a simulation on a single runtime container

1. Install and initialize the Cloud SDK..

2. Clone the TensorFlow Federated repository.

$ git clone https://github.com/tensorflow/federated.git
$ cd "federated"

3. Run a single runtime container.

  1. Build a runtime container.
$ docker build \
    --network=host \
    --tag "<registry>/tff-runtime" \
    --file "tensorflow_federated/tools/runtime/container/latest.Dockerfile" \
    .
  1. Publish the runtime container.
$ docker push <registry>/tff-runtime
  1. Create a Compute Engine instance

    1. In the Cloud Console, go to the VM Instances page.

    2. Click Create instance.

    3. In the Firewall section, select Allow HTTP traffic and Allow HTTPS traffic.

    4. Click Create to create the instance.

  2. ssh into the instance.

$ gcloud compute ssh <instance>
  1. Run the runtime container in the background.
$ docker run \
    --detach \
    --name=tff-runtime \
    --publish=8000:8000 \
    <registry>/tff-runtime
  1. Exit the instance.
$ exit
  1. Get the internal IP address of the instance.

    This is used later as a parameter to our test script.

$ gcloud compute instances describe <instance> \
    --format='get(networkInterfaces[0].networkIP)'

4. Run a simulation on a client container.

  1. Build a client container.
$ docker build \
    --network=host \
    --tag "<registry>/tff-client" \
    --file "tensorflow_federated/tools/client/latest.Dockerfile" \
    .
  1. Publish the client container.
$ docker push <registry>/tff-client
  1. Create a Compute Engine instance

    1. In the Cloud Console, go to the VM Instances page.

    2. Click Create instance.

    3. In the Firewall section, select Allow HTTP traffic and Allow HTTPS traffic.

    4. Click Create to create the instance.

  2. Copy your experiement to the Compute Engine instance.

$ gcloud compute scp \
    "tensorflow_federated/tools/client/test.py" \
    <instance>:~
  1. ssh into the instance.
$ gcloud compute ssh <instance>
  1. Run the client container interactively.

    The string "Hello World" should print to the terminal.

$ docker run \
    --interactive \
    --tty \
    --name=tff-client \
    --volume ~:/simulation \
    --workdir /simulation \
    <registry>/tff-client \
    bash
  1. Run the Python script.

    Using the internal IP address of the instance running the runtime container.

$ python3 test.py --host '<IP address>'
  1. Exit the container.
$ exit
  1. Exit the instance.
$ exit