Missed TensorFlow Dev Summit? Check out the video playlist. Watch recordings

Setup simulations with TFF on GCP

This tutorial will describe how to do the steps required for the setup high-performance simulations with TFF on GCP.

  1. Install and initialize the Cloud SDK..

  2. Run a simulation on a single runtime container.

    1. Start a single runtime container.

      1. Create a Compute Engine instance.

      2. ssh into the instance.

$ gcloud compute ssh <instance>
    1.  Run the runtime container in the background.
$ docker run \
    --detach \
    --name=runtime \
    --publish=8000:8000 \
    gcr.io/tensorflow-federated/runtime
    1.  Exit the instance.
$ exit
    1.  Get the internal IP address of the instance.

        This is used later as a parameter to our test script.
$ gcloud compute instances describe <instance> \
    --format='get(networkInterfaces[0].networkIP)'
1.  Start and run a simulation on a client container.

    1.  [Create a Compute Engine instance](https://cloud.google.com/endpoints/docs/grpc/get-started-compute-engine-docker#create_vm).

    1.  Copy your experiement to the Compute Engine instance.
$ gcloud compute scp \
    "tensorflow_federated/tools/client/test.py" \
    <instance>:~
    1.  `ssh` into the instance.
$ gcloud compute ssh <instance>
    1.  Run the client container interactively.

        The string "Hello World" should print to the terminal.
$ docker run \
    --interactive \
    --tty \
    --name=client \
    --volume ~:/simulation \
    --workdir /simulation \
    gcr.io/tensorflow-federated/client \
    bash
    1.  Run the Python script.

        Using the internal IP address of the instance running the runtime
        container.
$ python3 test.py --host '<internal IP address>'
    1.  Exit the container.
$ exit
    1.  Exit the instance.
$ exit
  1. Configure runtime and client images from source.

    If you wanted to run a simulation using TFF from source instead of a released version of TFF, you need to: build the runtime and client images from source; publish those images to your own container registry; and finally create the runtime and client containers using those images instead of the released images that we provide. See the Container Registry documentation for more information.

    1. Configure a runtime image.
$ bazel run //tensorflow_federated/tools/runtime/gcp:build_image
$ bazel run //tensorflow_federated/tools/runtime/gcp:publish_image -- \
    <runtime registry>
1.  Configure a client image.
$ bazel run //tensorflow_federated/tools/client:build_image
$ bazel run //tensorflow_federated/tools/client:publish_image -- \
    <client registry>