View source on GitHub |
Cross device communication implementation.
AUTO
: Automatically chosen by Tensorflow.RING
: TensorFlow's ring algorithms for all-reduce and all-gather.NCCL
: NVIDIA®'s NCCL library. This is now only used for all-reduce on GPUs; all-reduce on CPU, all-gather and broadcast fallbacks to RING.
Class Variables | |
---|---|
AUTO |
<CommunicationImplementation.AUTO: 'AUTO'>
|
NCCL |
<CommunicationImplementation.NCCL: 'NCCL'>
|
RING |
<CommunicationImplementation.RING: 'RING'>
|