Post-training float16 quantization

View on TensorFlow.org Run in Google Colab View source on GitHub

Overview

TensorFlow Lite now supports converting weights to 16-bit floating point values during model conversion from TensorFlow to TensorFlow Lite's flat buffer format. This results in a 2x reduction in model size. Some harware, like GPUs, can compute natively in this reduced precision arithmetic, realizing a speedup over traditional floating point execution. The Tensorflow Lite GPU delegate can be configured to run in this way. However, a model converted to float16 weights can still run on the CPU without additional modification: the float16 weights are upsampled to float32 prior to the first inference. This permits a significant reduction in model size in exchange for a minimal impacts to latency and accuracy.

In this tutorial, you train an MNIST model from scratch, check its accuracy in TensorFlow, and then convert the saved model into a Tensorflow Lite flatbuffer with float16 quantization. Finally, check the accuracy of the converted model and compare it to the original saved model. The training script, mnist.py, is available from the TensorFlow official MNIST tutorial.

Build an MNIST model

Setup

! pip uninstall -y tensorflow
! pip install -q -U tf-nightly
WARNING: Skipping tensorflow as it is not installed.
import tensorflow as tf
tf.enable_eager_execution()

import numpy as np

tf.logging.set_verbosity(tf.logging.DEBUG)
WARNING:tensorflow:

  TensorFlow's `tf-nightly` package will soon be updated to TensorFlow 2.0.

  Please upgrade your code to TensorFlow 2.0:
    * https://www.tensorflow.org/beta/guide/migration_guide

  Or install the latest stable TensorFlow 1.X release:
    * `pip install -U "tensorflow==1.*"`

  Otherwise your code may be broken by the change.

  
! git clone --depth 1 https://github.com/tensorflow/models
Cloning into 'models'...
remote: Enumerating objects: 3230, done.
remote: Counting objects: 100% (3230/3230), done.
remote: Compressing objects: 100% (2737/2737), done.
remote: Total 3230 (delta 598), reused 2028 (delta 417), pack-reused 0
Receiving objects: 100% (3230/3230), 370.70 MiB | 41.76 MiB/s, done.
Resolving deltas: 100% (598/598), done.
Checking out files: 100% (3055/3055), done.
tf.lite.constants.FLOAT16
tf.float16
import sys
import os

if sys.version_info.major >= 3:
    import pathlib
else:
    import pathlib2 as pathlib

# Add `models` to the python path.
models_path = os.path.join(os.getcwd(), "models")
sys.path.append(models_path)

Train and export the model

saved_models_root = "/tmp/mnist_saved_model"
# The above path addition is not visible to subprocesses, add the path for the subprocess as well.
!PYTHONPATH={models_path} python models/official/mnist/mnist.py --train_epochs=1 --export_dir {saved_models_root} --data_format=channels_last
WARNING:tensorflow:

  TensorFlow's `tf-nightly` package will soon be updated to TensorFlow 2.0.

  Please upgrade your code to TensorFlow 2.0:
    * https://www.tensorflow.org/beta/guide/migration_guide

  Or install the latest stable TensorFlow 1.X release:
    * `pip install -U "tensorflow==1.*"`

  Otherwise your code may be broken by the change.

  
WARNING:tensorflow:From models/official/mnist/mnist.py:242: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

WARNING:tensorflow:From models/official/mnist/mnist.py:168: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

W1001 10:54:08.678114 140582814680832 module_wrapper.py:137] From models/official/mnist/mnist.py:168: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

INFO:tensorflow:Initializing RunConfig with distribution strategies.
I1001 10:54:08.679298 140582814680832 run_config.py:566] Initializing RunConfig with distribution strategies.
INFO:tensorflow:Not using Distribute Coordinator.
I1001 10:54:08.679556 140582814680832 estimator_training.py:167] Not using Distribute Coordinator.
INFO:tensorflow:Using config: {'_is_chief': True, '_evaluation_master': '', '_save_checkpoints_secs': 600, '_protocol': None, '_session_creation_timeout_secs': 7200, '_tf_random_seed': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7fdbcaaee0b8>, '_experimental_max_worker_delay_secs': None, '_experimental_distribute': None, '_eval_distribute': None, '_master': '', '_global_id_in_cluster': 0, '_num_worker_replicas': 1, '_keep_checkpoint_every_n_hours': 10000, '_num_ps_replicas': 0, '_service': None, '_distribute_coordinator_mode': None, '_session_config': allow_soft_placement: true
, '_save_summary_steps': 100, '_task_id': 0, '_device_fn': None, '_log_step_count_steps': 100, '_model_dir': '/tmp/mnist_model', '_keep_checkpoint_max': 5, '_save_checkpoints_steps': None, '_train_distribute': <tensorflow.python.distribute.one_device_strategy.OneDeviceStrategyV1 object at 0x7fdbcaaee208>, '_task_type': 'worker'}
I1001 10:54:08.680268 140582814680832 estimator.py:212] Using config: {'_is_chief': True, '_evaluation_master': '', '_save_checkpoints_secs': 600, '_protocol': None, '_session_creation_timeout_secs': 7200, '_tf_random_seed': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7fdbcaaee0b8>, '_experimental_max_worker_delay_secs': None, '_experimental_distribute': None, '_eval_distribute': None, '_master': '', '_global_id_in_cluster': 0, '_num_worker_replicas': 1, '_keep_checkpoint_every_n_hours': 10000, '_num_ps_replicas': 0, '_service': None, '_distribute_coordinator_mode': None, '_session_config': allow_soft_placement: true
, '_save_summary_steps': 100, '_task_id': 0, '_device_fn': None, '_log_step_count_steps': 100, '_model_dir': '/tmp/mnist_model', '_keep_checkpoint_max': 5, '_save_checkpoints_steps': None, '_train_distribute': <tensorflow.python.distribute.one_device_strategy.OneDeviceStrategyV1 object at 0x7fdbcaaee208>, '_task_type': 'worker'}
WARNING:tensorflow:From /tmpfs/src/temp/tensorflow/lite/g3doc/performance/models/official/mnist/dataset.py:65: The name tf.gfile.Exists is deprecated. Please use tf.io.gfile.exists instead.

W1001 10:54:08.682126 140582814680832 module_wrapper.py:137] From /tmpfs/src/temp/tensorflow/lite/g3doc/performance/models/official/mnist/dataset.py:65: The name tf.gfile.Exists is deprecated. Please use tf.io.gfile.exists instead.

WARNING:tensorflow:Entity <function dataset.<locals>.decode_image at 0x7fdbcaae3c80> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: No module named 'tensorflow_estimator.contrib'
W1001 10:54:09.365935 140582814680832 ag_logging.py:146] Entity <function dataset.<locals>.decode_image at 0x7fdbcaae3c80> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: No module named 'tensorflow_estimator.contrib'
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
W1001 10:54:09.520537 140582814680832 deprecation.py:506] From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
INFO:tensorflow:Calling model_fn.
I1001 10:54:09.524553 140582814680832 estimator.py:1147] Calling model_fn.
WARNING:tensorflow:From models/official/mnist/mnist.py:127: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

W1001 10:54:09.646960 140582814680832 module_wrapper.py:137] From models/official/mnist/mnist.py:127: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

WARNING:tensorflow:From models/official/mnist/mnist.py:130: The name tf.losses.sparse_softmax_cross_entropy is deprecated. Please use tf.compat.v1.losses.sparse_softmax_cross_entropy instead.

W1001 10:54:09.685475 140582814680832 module_wrapper.py:137] From models/official/mnist/mnist.py:130: The name tf.losses.sparse_softmax_cross_entropy is deprecated. Please use tf.compat.v1.losses.sparse_softmax_cross_entropy instead.

WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/python/ops/losses/losses_impl.py:121: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
W1001 10:54:09.696224 140582814680832 deprecation.py:323] From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/python/ops/losses/losses_impl.py:121: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From models/official/mnist/mnist.py:131: The name tf.metrics.accuracy is deprecated. Please use tf.compat.v1.metrics.accuracy instead.

W1001 10:54:09.705745 140582814680832 module_wrapper.py:137] From models/official/mnist/mnist.py:131: The name tf.metrics.accuracy is deprecated. Please use tf.compat.v1.metrics.accuracy instead.

WARNING:tensorflow:From models/official/mnist/mnist.py:140: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead.

W1001 10:54:09.730048 140582814680832 module_wrapper.py:137] From models/official/mnist/mnist.py:140: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead.

INFO:tensorflow:Done calling model_fn.
I1001 10:54:09.948710 140582814680832 estimator.py:1149] Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
I1001 10:54:09.986137 140582814680832 basic_session_run_hooks.py:541] Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
I1001 10:54:10.151815 140582814680832 monitored_session.py:240] Graph was finalized.
2019-10-01 10:54:10.152323: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-10-01 10:54:10.157943: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2200000000 Hz
2019-10-01 10:54:10.158279: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x72e71d0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2019-10-01 10:54:10.158321: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
INFO:tensorflow:Restoring parameters from /tmp/mnist_model/model.ckpt-600
I1001 10:54:10.161282 140582814680832 saver.py:1284] Restoring parameters from /tmp/mnist_model/model.ckpt-600
2019-10-01 10:54:10.174610: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
IteratorToStringHandle: CPU XLA_CPU 
IteratorV2: CPU XLA_CPU 
IteratorGetNext: CPU XLA_CPU 
MakeIterator: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  IteratorV2 (IteratorV2) /replica:0/task:0/device:GPU:0
  MakeIterator (MakeIterator) /replica:0/task:0/device:GPU:0
  IteratorToStringHandle (IteratorToStringHandle) /replica:0/task:0/device:GPU:0
  IteratorGetNext (IteratorGetNext) /replica:0/task:0/device:GPU:0

2019-10-01 10:54:10.174713: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
AssignAddVariableOp: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
Const: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  global_step/Initializer/zeros (Const) 
  global_step (VarHandleOp) /replica:0/task:0/device:GPU:0
  global_step/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  global_step/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  global_step/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Identity/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/AssignAddVariableOp (AssignAddVariableOp) /replica:0/task:0/device:GPU:0
  Adam/ReadVariableOp_4 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp (VarIsInitializedOp) 
  save/AssignVariableOp_26 (AssignVariableOp) /replica:0/task:0/device:GPU:0

2019-10-01 10:54:10.174853: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
ResourceApplyAdam: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
RandomUniform: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
Const: CPU XLA_CPU 
Mul: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 
Sub: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 
Add: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  conv2d/kernel/Initializer/random_uniform/shape (Const) 
  conv2d/kernel/Initializer/random_uniform/min (Const) 
  conv2d/kernel/Initializer/random_uniform/max (Const) 
  conv2d/kernel/Initializer/random_uniform/RandomUniform (RandomUniform) 
  conv2d/kernel/Initializer/random_uniform/sub (Sub) 
  conv2d/kernel/Initializer/random_uniform/mul (Mul) 
  conv2d/kernel/Initializer/random_uniform (Add) 
  conv2d/kernel (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/Conv2D/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  sequential/conv2d/Conv2D/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Adam/Initializer/zeros (Const) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Adam (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Adam/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Adam/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Adam/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Adam_1/Initializer/zeros (Const) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Adam_1 (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Adam_1/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Adam_1/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Adam_1/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d/kernel/ResourceApplyAdam (ResourceApplyAdam) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_1 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_11 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_12 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp_1 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_11 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_12 (VarIsInitializedOp) 
  save/AssignVariableOp_5 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_6 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_7 (AssignVariableOp) /replica:0/task:0/device:GPU:0

2019-10-01 10:54:10.175059: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
Mul: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 
Const: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 
ResourceApplyAdam: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  conv2d/bias/Initializer/zeros (Const) 
  conv2d/bias (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/BiasAdd/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  sequential/conv2d/BiasAdd/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  beta1_power/Initializer/initial_value (Const) /replica:0/task:0/device:GPU:0
  beta1_power (VarHandleOp) /replica:0/task:0/device:GPU:0
  beta1_power/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  beta1_power/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  beta1_power/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  beta2_power/Initializer/initial_value (Const) /replica:0/task:0/device:GPU:0
  beta2_power (VarHandleOp) /replica:0/task:0/device:GPU:0
  beta2_power/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  beta2_power/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  beta2_power/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/Adam/Initializer/zeros (Const) /replica:0/task:0/device:GPU:0
  conv2d/bias/Adam (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/Adam/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/Adam/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/Adam/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/Adam_1/Initializer/zeros (Const) /replica:0/task:0/device:GPU:0
  conv2d/bias/Adam_1 (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/Adam_1/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/Adam_1/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/Adam_1/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d/kernel/ResourceApplyAdam/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d/kernel/ResourceApplyAdam/ReadVariableOp_1 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d/bias/ResourceApplyAdam/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d/bias/ResourceApplyAdam/ReadVariableOp_1 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d/bias/ResourceApplyAdam (ResourceApplyAdam) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d_1/kernel/ResourceApplyAdam/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d_1/kernel/ResourceApplyAdam/ReadVariableOp_1 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d_1/bias/ResourceApplyAdam/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d_1/bias/ResourceApplyAdam/ReadVariableOp_1 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense/kernel/ResourceApplyAdam/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense/kernel/ResourceApplyAdam/ReadVariableOp_1 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense/bias/ResourceApplyAdam/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense/bias/ResourceApplyAdam/ReadVariableOp_1 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense_1/kernel/ResourceApplyAdam/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense_1/kernel/ResourceApplyAdam/ReadVariableOp_1 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense_1/bias/ResourceApplyAdam/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense_1/bias/ResourceApplyAdam/ReadVariableOp_1 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/mul (Mul) /replica:0/task:0/device:GPU:0
  Adam/AssignVariableOp (AssignVariableOp) /replica:0/task:0/device:GPU:0
  Adam/ReadVariableOp_1 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/ReadVariableOp_2 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/mul_1 (Mul) /replica:0/task:0/device:GPU:0
  Adam/AssignVariableOp_1 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  Adam/ReadVariableOp_3 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_2 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_9 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_10 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_13 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_14 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp_2 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_9 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_10 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_13 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_14 (VarIsInitializedOp) 
  save/AssignVariableOp (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_1 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_2 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_3 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_4 (AssignVariableOp) /replica:0/task:0/device:GPU:0

2019-10-01 10:54:10.175420: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
ResourceApplyAdam: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
RandomUniform: CPU XLA_CPU 
Fill: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
Const: CPU XLA_CPU 
Mul: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 
Sub: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 
Add: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  conv2d_1/kernel/Initializer/random_uniform/shape (Const) 
  conv2d_1/kernel/Initializer/random_uniform/min (Const) 
  conv2d_1/kernel/Initializer/random_uniform/max (Const) 
  conv2d_1/kernel/Initializer/random_uniform/RandomUniform (RandomUniform) 
  conv2d_1/kernel/Initializer/random_uniform/sub (Sub) 
  conv2d_1/kernel/Initializer/random_uniform/mul (Mul) 
  conv2d_1/kernel/Initializer/random_uniform (Add) 
  conv2d_1/kernel (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/Conv2D/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  sequential/conv2d_1/Conv2D/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam/Initializer/zeros/shape_as_tensor (Const) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam/Initializer/zeros/Const (Const) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam/Initializer/zeros (Fill) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam_1/Initializer/zeros/shape_as_tensor (Const) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam_1/Initializer/zeros/Const (Const) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam_1/Initializer/zeros (Fill) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam_1 (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam_1/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam_1/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam_1/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d_1/kernel/ResourceApplyAdam (ResourceApplyAdam) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_3 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_15 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_16 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp_3 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_15 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_16 (VarIsInitializedOp) 
  save/AssignVariableOp_11 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_12 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_13 (AssignVariableOp) /replica:0/task:0/device:GPU:0

2019-10-01 10:54:10.175701: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
ResourceApplyAdam: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
Const: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  conv2d_1/bias/Initializer/zeros (Const) 
  conv2d_1/bias (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/BiasAdd/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  sequential/conv2d_1/BiasAdd/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Adam/Initializer/zeros (Const) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Adam (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Adam/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Adam/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Adam/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Adam_1/Initializer/zeros (Const) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Adam_1 (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Adam_1/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Adam_1/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Adam_1/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d_1/bias/ResourceApplyAdam (ResourceApplyAdam) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_4 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_17 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_18 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp_4 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_17 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_18 (VarIsInitializedOp) 
  save/AssignVariableOp_8 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_9 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_10 (AssignVariableOp) /replica:0/task:0/device:GPU:0

2019-10-01 10:54:10.175960: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
ResourceApplyAdam: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
RandomUniform: CPU XLA_CPU 
Fill: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
Const: CPU XLA_CPU 
Mul: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 
Sub: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 
Add: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  dense/kernel/Initializer/random_uniform/shape (Const) 
  dense/kernel/Initializer/random_uniform/min (Const) 
  dense/kernel/Initializer/random_uniform/max (Const) 
  dense/kernel/Initializer/random_uniform/RandomUniform (RandomUniform) 
  dense/kernel/Initializer/random_uniform/sub (Sub) 
  dense/kernel/Initializer/random_uniform/mul (Mul) 
  dense/kernel/Initializer/random_uniform (Add) 
  dense/kernel (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense/kernel/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense/kernel/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense/kernel/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense/MatMul/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  sequential/dense/MatMul/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam/Initializer/zeros/shape_as_tensor (Const) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam/Initializer/zeros/Const (Const) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam/Initializer/zeros (Fill) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam_1/Initializer/zeros/shape_as_tensor (Const) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam_1/Initializer/zeros/Const (Const) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam_1/Initializer/zeros (Fill) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam_1 (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam_1/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam_1/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam_1/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense/kernel/ResourceApplyAdam (ResourceApplyAdam) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_5 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_19 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_20 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp_5 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_19 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_20 (VarIsInitializedOp) 
  save/AssignVariableOp_17 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_18 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_19 (AssignVariableOp) /replica:0/task:0/device:GPU:0

2019-10-01 10:54:10.176244: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
ResourceApplyAdam: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 
Fill: CPU XLA_CPU 
Const: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  dense/bias/Initializer/zeros/shape_as_tensor (Const) 
  dense/bias/Initializer/zeros/Const (Const) 
  dense/bias/Initializer/zeros (Fill) 
  dense/bias (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense/bias/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense/bias/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense/bias/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense/BiasAdd/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  sequential/dense/BiasAdd/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense/bias/Adam/Initializer/zeros/shape_as_tensor (Const) /replica:0/task:0/device:GPU:0
  dense/bias/Adam/Initializer/zeros/Const (Const) /replica:0/task:0/device:GPU:0
  dense/bias/Adam/Initializer/zeros (Fill) /replica:0/task:0/device:GPU:0
  dense/bias/Adam (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense/bias/Adam/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense/bias/Adam/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense/bias/Adam/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense/bias/Adam_1/Initializer/zeros/shape_as_tensor (Const) /replica:0/task:0/device:GPU:0
  dense/bias/Adam_1/Initializer/zeros/Const (Const) /replica:0/task:0/device:GPU:0
  dense/bias/Adam_1/Initializer/zeros (Fill) /replica:0/task:0/device:GPU:0
  dense/bias/Adam_1 (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense/bias/Adam_1/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense/bias/Adam_1/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense/bias/Adam_1/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense/bias/ResourceApplyAdam (ResourceApplyAdam) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_6 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_21 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_22 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp_6 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_21 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_22 (VarIsInitializedOp) 
  save/AssignVariableOp_14 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_15 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_16 (AssignVariableOp) /replica:0/task:0/device:GPU:0

2019-10-01 10:54:10.176609: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
ResourceApplyAdam: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
RandomUniform: CPU XLA_CPU 
Fill: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
Const: CPU XLA_CPU 
Mul: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 
Sub: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 
Add: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  dense_1/kernel/Initializer/random_uniform/shape (Const) 
  dense_1/kernel/Initializer/random_uniform/min (Const) 
  dense_1/kernel/Initializer/random_uniform/max (Const) 
  dense_1/kernel/Initializer/random_uniform/RandomUniform (RandomUniform) 
  dense_1/kernel/Initializer/random_uniform/sub (Sub) 
  dense_1/kernel/Initializer/random_uniform/mul (Mul) 
  dense_1/kernel/Initializer/random_uniform (Add) 
  dense_1/kernel (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/MatMul/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  sequential/dense_1/MatMul/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam/Initializer/zeros/shape_as_tensor (Const) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam/Initializer/zeros/Const (Const) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam/Initializer/zeros (Fill) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam_1/Initializer/zeros/shape_as_tensor (Const) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam_1/Initializer/zeros/Const (Const) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam_1/Initializer/zeros (Fill) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam_1 (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam_1/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam_1/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam_1/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense_1/kernel/ResourceApplyAdam (ResourceApplyAdam) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_7 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_23 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_24 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp_7 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_23 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_24 (VarIsInitializedOp) 
  save/AssignVariableOp_23 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_24 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_25 (AssignVariableOp) /replica:0/task:0/device:GPU:0

2019-10-01 10:54:10.176913: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
ResourceApplyAdam: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
Const: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  dense_1/bias/Initializer/zeros (Const) 
  dense_1/bias (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/BiasAdd/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  sequential/dense_1/BiasAdd/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/Adam/Initializer/zeros (Const) /replica:0/task:0/device:GPU:0
  dense_1/bias/Adam (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/Adam/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/Adam/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/Adam/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/Adam_1/Initializer/zeros (Const) /replica:0/task:0/device:GPU:0
  dense_1/bias/Adam_1 (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/Adam_1/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/Adam_1/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/Adam_1/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense_1/bias/ResourceApplyAdam (ResourceApplyAdam) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_8 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_25 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_26 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp_8 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_25 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_26 (VarIsInitializedOp) 
  save/AssignVariableOp_20 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_21 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_22 (AssignVariableOp) /replica:0/task:0/device:GPU:0

2019-10-01 10:54:10.179131: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
AssignAddVariableOp: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
Const: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  accuracy/total/Initializer/zeros (Const) 
  accuracy/total (VarHandleOp) /replica:0/task:0/device:GPU:0
  accuracy/total/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  accuracy/total/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  accuracy/total/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  accuracy/AssignAddVariableOp (AssignAddVariableOp) /replica:0/task:0/device:GPU:0
  accuracy/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  accuracy/value/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  accuracy/update_op/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp_27 (VarIsInitializedOp) 

2019-10-01 10:54:10.179239: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
AssignAddVariableOp: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
Const: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  accuracy/count/Initializer/zeros (Const) 
  accuracy/count (VarHandleOp) /replica:0/task:0/device:GPU:0
  accuracy/count/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  accuracy/count/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  accuracy/count/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  accuracy/AssignAddVariableOp_1 (AssignAddVariableOp) /replica:0/task:0/device:GPU:0
  accuracy/ReadVariableOp_1 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  accuracy/Maximum/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  accuracy/Maximum_1/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp_28 (VarIsInitializedOp) 

WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/python/training/saver.py:1069: get_checkpoint_mtimes (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file utilities to get mtimes.
W1001 10:54:10.244365 140582814680832 deprecation.py:323] From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/python/training/saver.py:1069: get_checkpoint_mtimes (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file utilities to get mtimes.
INFO:tensorflow:Running local_init_op.
I1001 10:54:10.280752 140582814680832 session_manager.py:500] Running local_init_op.
INFO:tensorflow:Done running local_init_op.
I1001 10:54:10.294211 140582814680832 session_manager.py:502] Done running local_init_op.
INFO:tensorflow:Saving checkpoints for 600 into /tmp/mnist_model/model.ckpt.
I1001 10:54:10.617438 140582814680832 basic_session_run_hooks.py:606] Saving checkpoints for 600 into /tmp/mnist_model/model.ckpt.
INFO:tensorflow:cross_entropy = 0.15431236, learning_rate = 1e-04, train_accuracy = 0.97
I1001 10:54:16.770719 140582814680832 basic_session_run_hooks.py:262] cross_entropy = 0.15431236, learning_rate = 1e-04, train_accuracy = 0.97
INFO:tensorflow:loss = 0.15431236, step = 600
I1001 10:54:16.771355 140582814680832 basic_session_run_hooks.py:262] loss = 0.15431236, step = 600
INFO:tensorflow:global_step/sec: 6.09133
I1001 10:54:33.187195 140582814680832 basic_session_run_hooks.py:692] global_step/sec: 6.09133
INFO:tensorflow:cross_entropy = 0.079344004, learning_rate = 1e-04, train_accuracy = 0.975 (16.418 sec)
I1001 10:54:33.188281 140582814680832 basic_session_run_hooks.py:260] cross_entropy = 0.079344004, learning_rate = 1e-04, train_accuracy = 0.975 (16.418 sec)
INFO:tensorflow:loss = 0.079344004, step = 700 (16.417 sec)
I1001 10:54:33.188595 140582814680832 basic_session_run_hooks.py:260] loss = 0.079344004, step = 700 (16.417 sec)
INFO:tensorflow:global_step/sec: 6.42456
I1001 10:54:48.752461 140582814680832 basic_session_run_hooks.py:692] global_step/sec: 6.42456
INFO:tensorflow:cross_entropy = 0.13019127, learning_rate = 1e-04, train_accuracy = 0.9766667 (15.565 sec)
I1001 10:54:48.753443 140582814680832 basic_session_run_hooks.py:260] cross_entropy = 0.13019127, learning_rate = 1e-04, train_accuracy = 0.9766667 (15.565 sec)
INFO:tensorflow:loss = 0.13019127, step = 800 (15.565 sec)
I1001 10:54:48.753735 140582814680832 basic_session_run_hooks.py:260] loss = 0.13019127, step = 800 (15.565 sec)
INFO:tensorflow:global_step/sec: 6.43864
I1001 10:55:04.283711 140582814680832 basic_session_run_hooks.py:692] global_step/sec: 6.43864
INFO:tensorflow:cross_entropy = 0.079699785, learning_rate = 1e-04, train_accuracy = 0.975 (15.531 sec)
I1001 10:55:04.284659 140582814680832 basic_session_run_hooks.py:260] cross_entropy = 0.079699785, learning_rate = 1e-04, train_accuracy = 0.975 (15.531 sec)
INFO:tensorflow:loss = 0.079699785, step = 900 (15.531 sec)
I1001 10:55:04.285107 140582814680832 basic_session_run_hooks.py:260] loss = 0.079699785, step = 900 (15.531 sec)
INFO:tensorflow:global_step/sec: 6.39906
I1001 10:55:19.910967 140582814680832 basic_session_run_hooks.py:692] global_step/sec: 6.39906
INFO:tensorflow:cross_entropy = 0.040272422, learning_rate = 1e-04, train_accuracy = 0.978 (15.627 sec)
I1001 10:55:19.911880 140582814680832 basic_session_run_hooks.py:260] cross_entropy = 0.040272422, learning_rate = 1e-04, train_accuracy = 0.978 (15.627 sec)
INFO:tensorflow:loss = 0.040272422, step = 1000 (15.627 sec)
I1001 10:55:19.912255 140582814680832 basic_session_run_hooks.py:260] loss = 0.040272422, step = 1000 (15.627 sec)
INFO:tensorflow:global_step/sec: 6.57013
I1001 10:55:35.131398 140582814680832 basic_session_run_hooks.py:692] global_step/sec: 6.57013
INFO:tensorflow:cross_entropy = 0.055716805, learning_rate = 1e-04, train_accuracy = 0.97833335 (15.221 sec)
I1001 10:55:35.132503 140582814680832 basic_session_run_hooks.py:260] cross_entropy = 0.055716805, learning_rate = 1e-04, train_accuracy = 0.97833335 (15.221 sec)
INFO:tensorflow:loss = 0.055716805, step = 1100 (15.221 sec)
I1001 10:55:35.132779 140582814680832 basic_session_run_hooks.py:260] loss = 0.055716805, step = 1100 (15.221 sec)
INFO:tensorflow:Saving checkpoints for 1200 into /tmp/mnist_model/model.ckpt.
I1001 10:55:49.945619 140582814680832 basic_session_run_hooks.py:606] Saving checkpoints for 1200 into /tmp/mnist_model/model.ckpt.
INFO:tensorflow:Loss for final step: 0.09040752.
I1001 10:55:50.088479 140582814680832 estimator.py:371] Loss for final step: 0.09040752.
WARNING:tensorflow:From models/official/mnist/mnist.py:210: DatasetV1.make_one_shot_iterator (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `for ... in dataset:` to iterate over a dataset. If using `tf.estimator`, return the `Dataset` object directly from your input function. As a last resort, you can use `tf.compat.v1.data.make_one_shot_iterator(dataset)`.
W1001 10:55:50.208173 140582814680832 deprecation.py:323] From models/official/mnist/mnist.py:210: DatasetV1.make_one_shot_iterator (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `for ... in dataset:` to iterate over a dataset. If using `tf.estimator`, return the `Dataset` object directly from your input function. As a last resort, you can use `tf.compat.v1.data.make_one_shot_iterator(dataset)`.
INFO:tensorflow:Calling model_fn.
I1001 10:55:50.223560 140582814680832 estimator.py:1147] Calling model_fn.
INFO:tensorflow:Done calling model_fn.
I1001 10:55:50.397914 140582814680832 estimator.py:1149] Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2019-10-01T10:55:50Z
I1001 10:55:50.417824 140582814680832 evaluation.py:255] Starting evaluation at 2019-10-01T10:55:50Z
INFO:tensorflow:Graph was finalized.
I1001 10:55:50.496585 140582814680832 monitored_session.py:240] Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/mnist_model/model.ckpt-1200
I1001 10:55:50.497858 140582814680832 saver.py:1284] Restoring parameters from /tmp/mnist_model/model.ckpt-1200
INFO:tensorflow:Running local_init_op.
I1001 10:55:50.551896 140582814680832 session_manager.py:500] Running local_init_op.
INFO:tensorflow:Done running local_init_op.
I1001 10:55:50.564892 140582814680832 session_manager.py:502] Done running local_init_op.
INFO:tensorflow:Finished evaluation at 2019-10-01-10:55:55
I1001 10:55:55.769824 140582814680832 evaluation.py:275] Finished evaluation at 2019-10-01-10:55:55
INFO:tensorflow:Saving dict for global step 1200: accuracy = 0.9798, global_step = 1200, loss = 0.061782017
I1001 10:55:55.770102 140582814680832 estimator.py:2049] Saving dict for global step 1200: accuracy = 0.9798, global_step = 1200, loss = 0.061782017
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1200: /tmp/mnist_model/model.ckpt-1200
I1001 10:55:55.822969 140582814680832 estimator.py:2109] Saving 'checkpoint_path' summary for global step 1200: /tmp/mnist_model/model.ckpt-1200

Evaluation results:
    {'loss': 0.061782017, 'accuracy': 0.9798, 'global_step': 1200}

WARNING:tensorflow:From models/official/mnist/mnist.py:234: Estimator.export_savedmodel (from tensorflow_estimator.python.estimator.estimator) is deprecated and will be removed in a future version.
Instructions for updating:
This function has been renamed, use `export_saved_model` instead.
W1001 10:55:55.824698 140582814680832 deprecation.py:323] From models/official/mnist/mnist.py:234: Estimator.export_savedmodel (from tensorflow_estimator.python.estimator.estimator) is deprecated and will be removed in a future version.
Instructions for updating:
This function has been renamed, use `export_saved_model` instead.
INFO:tensorflow:Calling model_fn.
I1001 10:55:55.831617 140582814680832 estimator.py:1147] Calling model_fn.
INFO:tensorflow:Done calling model_fn.
I1001 10:55:55.978631 140582814680832 estimator.py:1149] Done calling model_fn.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/python/saved_model/signature_def_utils_impl.py:201: build_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.utils.build_tensor_info or tf.compat.v1.saved_model.build_tensor_info.
W1001 10:55:55.978953 140582814680832 deprecation.py:323] From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/python/saved_model/signature_def_utils_impl.py:201: build_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.utils.build_tensor_info or tf.compat.v1.saved_model.build_tensor_info.
INFO:tensorflow:Signatures INCLUDED in export for Classify: None
I1001 10:55:55.979406 140582814680832 export_utils.py:170] Signatures INCLUDED in export for Classify: None
INFO:tensorflow:Signatures INCLUDED in export for Predict: ['serving_default', 'classify']
I1001 10:55:55.979598 140582814680832 export_utils.py:170] Signatures INCLUDED in export for Predict: ['serving_default', 'classify']
INFO:tensorflow:Signatures INCLUDED in export for Regress: None
I1001 10:55:55.979742 140582814680832 export_utils.py:170] Signatures INCLUDED in export for Regress: None
INFO:tensorflow:Signatures INCLUDED in export for Train: None
I1001 10:55:55.979890 140582814680832 export_utils.py:170] Signatures INCLUDED in export for Train: None
INFO:tensorflow:Signatures INCLUDED in export for Eval: None
I1001 10:55:55.980029 140582814680832 export_utils.py:170] Signatures INCLUDED in export for Eval: None
INFO:tensorflow:Restoring parameters from /tmp/mnist_model/model.ckpt-1200
I1001 10:55:56.005295 140582814680832 saver.py:1284] Restoring parameters from /tmp/mnist_model/model.ckpt-1200
INFO:tensorflow:Assets added to graph.
I1001 10:55:56.029927 140582814680832 builder_impl.py:665] Assets added to graph.
INFO:tensorflow:No assets to write.
I1001 10:55:56.030166 140582814680832 builder_impl.py:460] No assets to write.
INFO:tensorflow:SavedModel written to: /tmp/mnist_saved_model/temp-b'1569952555'/saved_model.pb
I1001 10:55:56.086550 140582814680832 builder_impl.py:425] SavedModel written to: /tmp/mnist_saved_model/temp-b'1569952555'/saved_model.pb

For the example, you trained the model for just a single epoch, so it only trains to ~96% accuracy.

Convert to a TensorFlow Lite model

The savedmodel directory is named with a timestamp. Select the most recent one:

saved_model_dir = str(sorted(pathlib.Path(saved_models_root).glob("*"))[-1])
saved_model_dir
'/tmp/mnist_saved_model/1569952555'

Using the Python TFLiteConverter, the saved model can be converted into a TensorFlow Lite model.

First load the model using the TFLiteConverter:

converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
tflite_model = converter.convert()
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/lite/python/convert_saved_model.py:60: load (from tensorflow.python.saved_model.loader_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.loader.load or tf.compat.v1.saved_model.load. There will be a new function for importing SavedModels in Tensorflow 2.0.
INFO:tensorflow:Restoring parameters from /tmp/mnist_saved_model/1569952555/variables/variables
INFO:tensorflow:The given SavedModel MetaGraphDef contains SignatureDefs with the following keys: {'serving_default', 'classify'}
INFO:tensorflow:input tensors info: 
INFO:tensorflow:Tensor's key in saved_model's tensor_map: image
INFO:tensorflow: tensor name: Placeholder:0, shape: (-1, 28, 28), type: DT_FLOAT
INFO:tensorflow:output tensors info: 
INFO:tensorflow:Tensor's key in saved_model's tensor_map: probabilities
INFO:tensorflow: tensor name: Softmax:0, shape: (-1, 10), type: DT_FLOAT
INFO:tensorflow:Tensor's key in saved_model's tensor_map: classes
INFO:tensorflow: tensor name: ArgMax:0, shape: (-1), type: DT_INT64
INFO:tensorflow:Restoring parameters from /tmp/mnist_saved_model/1569952555/variables/variables
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/lite/python/util.py:249: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.convert_variables_to_constants`
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/python/framework/graph_util_impl.py:277: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.extract_sub_graph`
INFO:tensorflow:Froze 8 variables.
INFO:tensorflow:Converted 8 variables to const ops.

Write it out to a .tflite file:

tflite_models_dir = pathlib.Path("/tmp/mnist_tflite_models/")
tflite_models_dir.mkdir(exist_ok=True, parents=True)
tflite_model_file = tflite_models_dir/"mnist_model.tflite"
tflite_model_file.write_bytes(tflite_model)
13101352

To instead quantize the model to float16 on export, first set the optimizations flag to use default optimizations. Then specify that float16 is the supported type on the target platform:

tf.logging.set_verbosity(tf.logging.INFO)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_types = [tf.lite.constants.FLOAT16]

Finally, convert the model like usual. Note, by default the converted model will still use float input and outputs for invocation convenience.

tflite_fp16_model = converter.convert()
tflite_model_fp16_file = tflite_models_dir/"mnist_model_quant_f16.tflite"
tflite_model_fp16_file.write_bytes(tflite_fp16_model)
6552932

Note how the resulting file is approximately 1/2 the size.

!ls -lh {tflite_models_dir}
total 22M
-rw-rw-r-- 1 kbuilder kbuilder 6.3M Oct  1 10:56 mnist_model_quant_f16.tflite
-rw-rw-r-- 1 kbuilder kbuilder 3.2M Oct  1 10:51 mnist_model_quant.tflite
-rw-rw-r-- 1 kbuilder kbuilder  13M Oct  1 10:55 mnist_model.tflite

Run the TensorFlow Lite models

Run the TensorFlow Lite model using the Python TensorFlow Lite Interpreter.

Load the test data

First, let's load the MNIST test data to feed to the model:

_, mnist_test = tf.keras.datasets.mnist.load_data()
images, labels = tf.cast(mnist_test[0], tf.float32)/255.0, mnist_test[1]

mnist_ds = tf.data.Dataset.from_tensor_slices((images, labels)).batch(1)

Load the model into the interpreters

interpreter = tf.lite.Interpreter(model_path=str(tflite_model_file))
interpreter.allocate_tensors()
interpreter_fp16 = tf.lite.Interpreter(model_path=str(tflite_model_fp16_file))
interpreter_fp16.allocate_tensors()

Test the models on one image

for img, label in mnist_ds:
  break

interpreter.set_tensor(interpreter.get_input_details()[0]["index"], img)
interpreter.invoke()
predictions = interpreter.get_tensor(
    interpreter.get_output_details()[0]["index"])
import matplotlib.pylab as plt

plt.imshow(img[0])
template = "True:{true}, predicted:{predict}"
_ = plt.title(template.format(true= str(label[0].numpy()),
                              predict=str(predictions[0])))
plt.grid(False)
interpreter_fp16.set_tensor(
    interpreter_fp16.get_input_details()[0]["index"], img)
interpreter_fp16.invoke()
predictions = interpreter_fp16.get_tensor(
    interpreter_fp16.get_output_details()[0]["index"])
plt.imshow(img[0])
template = "True:{true}, predicted:{predict}"
_ = plt.title(template.format(true= str(label[0].numpy()),
                              predict=str(predictions[0])))
plt.grid(False)

png

Evaluate the models

def eval_model(interpreter, mnist_ds):
  total_seen = 0
  num_correct = 0

  input_index = interpreter.get_input_details()[0]["index"]
  output_index = interpreter.get_output_details()[0]["index"]
  for img, label in mnist_ds:
    total_seen += 1
    interpreter.set_tensor(input_index, img)
    interpreter.invoke()
    predictions = interpreter.get_tensor(output_index)
    if predictions == label.numpy():
      num_correct += 1

    if total_seen % 500 == 0:
      print("Accuracy after %i images: %f" %
            (total_seen, float(num_correct) / float(total_seen)))

  return float(num_correct) / float(total_seen)
# Create smaller dataset for demonstration purposes
mnist_ds_demo = mnist_ds.take(2000)

print(eval_model(interpreter, mnist_ds_demo))
Accuracy after 500 images: 0.984000
Accuracy after 1000 images: 0.980000
Accuracy after 1500 images: 0.976000
Accuracy after 2000 images: 0.975000
0.975

Repeat the evaluation on the float16 quantized model to obtain:

# NOTE: Colab runs on server CPUs. At the time of writing this, TensorFlow Lite
# doesn't have super optimized server CPU kernels. For this reason this may be
# slower than the above float interpreter. But for mobile CPUs, considerable
# speedup can be observed.
print(eval_model(interpreter_fp16, mnist_ds_demo))
Accuracy after 500 images: 0.984000
Accuracy after 1000 images: 0.980000
Accuracy after 1500 images: 0.976000
Accuracy after 2000 images: 0.975000
0.975

In this example, you have quantized a model to float16 with no difference in the accuracy.

It's also possible to evaluate the fp16 quantized model on the GPU. To perform all arithmetic with the reduced precision values, be sure to create the TfLiteGPUDelegateOptions struct in your app and set precision_loss_allowed to 1, like this:

//Prepare GPU delegate.
const TfLiteGpuDelegateOptions options = {
  .metadata = NULL,
  .compile_options = {
    .precision_loss_allowed = 1,  // FP16
    .preferred_gl_object_type = TFLITE_GL_OBJECT_TYPE_FASTEST,
    .dynamic_batch_enabled = 0,   // Not fully functional yet
  },
};

Detailed documentation on the TFLite GPU delegate and how to use it in your application can be found here