|TensorFlow 1 version||View source on GitHub|
ClusterResolver for system with Slurm workload manager.
Compat aliases for migration
See Migration guide for more details.
tf.distribute.cluster_resolver.SlurmClusterResolver( jobs=None, port_base=8888, gpus_per_node=None, gpus_per_task=None, tasks_per_node=None, auto_set_gpu=True, rpc_layer='grpc' )
This is an implementation of ClusterResolver for Slurm clusters. This allows the specification of jobs and task counts, number of tasks per node, number of GPUs on each node and number of GPUs for each task. It retrieves system attributes by Slurm environment variables, resolves allocated computing node names, constructs a cluster and returns a ClusterResolver object which can be used for distributed TensorFlow.
||Dictionary with job names as key and number of tasks in the job as value. Defaults to as many 'worker's as there are (Slurm) tasks.|
||The first port number to start with for processes on a node.|
||Number of GPUs available on each node. Defaults to the number of GPUs reported by nvidia-smi|
||Number of GPUs to be used for each task. Default is to evenly distribute the gpus_per_node to tasks_per_node.|
||Number of tasks running on each node. Can be an integer if the number of tasks per node is constant or a dictionary mapping hostnames to number of tasks on that node. If not set the Slurm environment is queried for the correct mapping.|
||Set the visible CUDA devices automatically while resolving the cluster by setting CUDA_VISIBLE_DEVICES environment variable. Defaults to True.|
||The protocol TensorFlow used to communicate between nodes. Defaults to 'grpc'.|
||If requested more GPUs per node then available or requested more tasks then assigned tasks or resolving missing values from the environment failed.|
Returns the current environment which TensorFlow is running in.
There are two possible return values, "google" (when TensorFlow is running in a Google-internal environment) or an empty string (when TensorFlow is running elsewhere).
If you are implementing a ClusterResolver that works in both the Google enviro