tfc.run

Runs your Tensorflow code in Google Cloud Platform.

tfc.run(
    entry_point=None, requirements_txt=None, docker_config='auto',
    distribution_strategy='auto', chief_config='auto',
    worker_config='auto', worker_count=0, entry_point_args=None,
    stream_logs=False, job_labels=None, service_account=None, **kwargs
)

Used in the notebooks

Used in the guide

entry_point Optional string. File path to the python file or iPython notebook that contains the TensorFlow code. Note this path must be in the current working directory tree. Example - 'train.py', 'training/mnist.py', 'mnist.ipynb' If entry_point is not provided, then

  • If you are in an iPython notebook environment, then the current notebook is taken as the entry_point.
  • Otherwise, the current python script is taken as the entry_point.
requirements_txt Optional string. File path to requirements.txt file containing additional pip dependencies if any. ie. a file with a list of pip dependency package names. Note this path must be in the current working directory tree. Example - 'requirements.txt', 'deps/reqs.txt'
docker_config Optional DockerConfig. Represents Docker related configuration for the run API.
  • image: Optional Docker image URI for the Docker image being built.
  • parent_image: Optional parent Docker image to use.
  • cache_from: Optional Docker image URI to be used as a cache when building the new Docker image.
  • image_build_bucket: Optional GCS bucket name to be used for building a Docker image via Google Cloud Build. Defaults to 'auto'. 'auto' maps to a default tfc.DockerConfig instance.
    • distribution_strategy 'auto' or None. Defaults to 'auto'. 'auto' means we will take care of creating a Tensorflow distribution strategy instance based on the machine configurations you have provided using the chief_config, worker_config and worker_count params.
  • If the number of workers > 0, we will use tf.distribute.experimental.MultiWorkerMirroredStrategy or tf.distribute.experimental.TPUStrategy based on the accelerator type.
  • If number of GPUs > 0, we will use tf.distribute.MirroredStrategy
  • Otherwise, we will use tf.distribute.OneDeviceStrategy If you have created a distribution strategy instance in your script already, please set distribution_strategy as None here. For example, if you are using tf.keras custom training loops, you will need to create a strategy in the script for distributing the dataset.
    • chief_config Optional MachineConfig that represents the configuration for the chief worker in a distribution cluster. Defaults to 'auto'. 'auto' maps to a standard gpu config such as COMMON_MACHINE_CONFIGS.T4_1X (8 cpu cores, 30GB memory, 1 Nvidia Tesla T4). For TPU strategy, chief_config refers to the config of the host that controls the TPU workers.
    worker_config Optional MachineConfig that represents the configuration for the general workers in a distribution cluster. Defaults to 'auto'. 'auto' maps to a standard gpu config such as COMMON_MACHINE_CONFIGS.T4_1X (8 cpu cores, 30GB memory, 1 Nvidia Tesla T4). For TPU strategy, worker_config should be a TPU config with 8 TPU cores (eg. COMMON_MACHINE_CONFIGS.TPU).
    worker_count Optional integer that represents the number of general workers in a distribution cluster. Defaults to 0. This count does not include the chief worker. For TPU strategy, worker_count should be set to 1.
    entry_point_args Optional list of strings. Defaults to None. Command line arguments to pass to the entry_point program.
    stream_logs Boolean flag which when enabled streams logs back from the cloud job.
    job_labels Dict of str: str. Labels to organize jobs. You can specify up to 64 key-value pairs in lowercase letters and numbers, where the first character must be lowercase letter. For more details see resource-labels
    service_account The email address of a user-managed service account to be used for training instead of the service account that AI Platform Training uses by default. see custom-service-account
    **kwargs Additional keyword arguments.

    A dictionary with two keys.'job_id' - the training job id and 'docker_image'- Docker image generated for the training job.