Post-training integer quantization

View on TensorFlow.org Run in Google Colab View source on GitHub

Overview

TensorFlow Lite now supports converting all model values (weights and activations) to 8-bit integers when converting from TensorFlow to TensorFlow Lite's flat buffer format. This results in a 4x reduction in model size and a 3 to 4x performance improvement on CPU performance. In addition, this fully quantized model can be consumed by integer-only hardware accelerators.

In contrast to post-training "on-the-fly" quantization—which stores only the weights as 8-bit integers—this technique statically quantizes all weights and activations during model conversion.

In this tutorial, you'll train an MNIST model from scratch, check its accuracy in TensorFlow, and then convert the saved model into a Tensorflow Lite flatbuffer with full quantization. Finally, you'll check the accuracy of the converted model and compare it to the original float model.

The training script, mnist.py, is available from the TensorFlow official MNIST tutorial.

Build an MNIST model

Setup

! pip uninstall -y tensorflow
! pip install -q -U tf-nightly
WARNING: Skipping tensorflow as it is not installed.
import tensorflow as tf
tf.enable_eager_execution()
WARNING:tensorflow:

  TensorFlow's `tf-nightly` package will soon be updated to TensorFlow 2.0.

  Please upgrade your code to TensorFlow 2.0:
    * https://www.tensorflow.org/beta/guide/migration_guide

  Or install the latest stable TensorFlow 1.X release:
    * `pip install -U "tensorflow==1.*"`

  Otherwise your code may be broken by the change.

  
! git clone --depth 1 https://github.com/tensorflow/models
Cloning into 'models'...
remote: Enumerating objects: 3230, done.
remote: Counting objects: 100% (3230/3230), done.
remote: Compressing objects: 100% (2737/2737), done.
remote: Total 3230 (delta 598), reused 2028 (delta 417), pack-reused 0
Receiving objects: 100% (3230/3230), 370.70 MiB | 41.77 MiB/s, done.
Resolving deltas: 100% (598/598), done.
Checking out files: 100% (3055/3055), done.
import sys
import os

if sys.version_info.major >= 3:
    import pathlib
else:
    import pathlib2 as pathlib

# Add `models` to the python path.
models_path = os.path.join(os.getcwd(), "models")
sys.path.append(models_path)

Train and export the model

saved_models_root = "/tmp/mnist_saved_model"
# The above path addition is not visible to subprocesses, add the path for the subprocess as well.
# Note: channels_last is required here or the conversion may fail. 
!PYTHONPATH={models_path} python models/official/mnist/mnist.py --train_epochs=1 --export_dir {saved_models_root} --data_format=channels_last
WARNING:tensorflow:

  TensorFlow's `tf-nightly` package will soon be updated to TensorFlow 2.0.

  Please upgrade your code to TensorFlow 2.0:
    * https://www.tensorflow.org/beta/guide/migration_guide

  Or install the latest stable TensorFlow 1.X release:
    * `pip install -U "tensorflow==1.*"`

  Otherwise your code may be broken by the change.

  
WARNING:tensorflow:From models/official/mnist/mnist.py:242: The name tf.logging.set_verbosity is deprecated. Please use tf.compat.v1.logging.set_verbosity instead.

WARNING:tensorflow:From models/official/mnist/mnist.py:168: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

W1001 10:57:29.504731 140568531904256 module_wrapper.py:137] From models/official/mnist/mnist.py:168: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.

INFO:tensorflow:Initializing RunConfig with distribution strategies.
I1001 10:57:29.505813 140568531904256 run_config.py:566] Initializing RunConfig with distribution strategies.
INFO:tensorflow:Not using Distribute Coordinator.
I1001 10:57:29.506027 140568531904256 estimator_training.py:167] Not using Distribute Coordinator.
INFO:tensorflow:Using config: {'_save_checkpoints_steps': None, '_num_ps_replicas': 0, '_evaluation_master': '', '_session_config': allow_soft_placement: true
, '_task_type': 'worker', '_keep_checkpoint_max': 5, '_save_checkpoints_secs': 600, '_experimental_max_worker_delay_secs': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7fd8775cc0b8>, '_session_creation_timeout_secs': 7200, '_service': None, '_is_chief': True, '_experimental_distribute': None, '_eval_distribute': None, '_tf_random_seed': None, '_model_dir': '/tmp/mnist_model', '_distribute_coordinator_mode': None, '_log_step_count_steps': 100, '_protocol': None, '_train_distribute': <tensorflow.python.distribute.one_device_strategy.OneDeviceStrategyV1 object at 0x7fd8775cc208>, '_keep_checkpoint_every_n_hours': 10000, '_num_worker_replicas': 1, '_save_summary_steps': 100, '_device_fn': None, '_master': '', '_task_id': 0, '_global_id_in_cluster': 0}
I1001 10:57:29.506593 140568531904256 estimator.py:212] Using config: {'_save_checkpoints_steps': None, '_num_ps_replicas': 0, '_evaluation_master': '', '_session_config': allow_soft_placement: true
, '_task_type': 'worker', '_keep_checkpoint_max': 5, '_save_checkpoints_secs': 600, '_experimental_max_worker_delay_secs': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7fd8775cc0b8>, '_session_creation_timeout_secs': 7200, '_service': None, '_is_chief': True, '_experimental_distribute': None, '_eval_distribute': None, '_tf_random_seed': None, '_model_dir': '/tmp/mnist_model', '_distribute_coordinator_mode': None, '_log_step_count_steps': 100, '_protocol': None, '_train_distribute': <tensorflow.python.distribute.one_device_strategy.OneDeviceStrategyV1 object at 0x7fd8775cc208>, '_keep_checkpoint_every_n_hours': 10000, '_num_worker_replicas': 1, '_save_summary_steps': 100, '_device_fn': None, '_master': '', '_task_id': 0, '_global_id_in_cluster': 0}
WARNING:tensorflow:From /tmpfs/src/temp/tensorflow/lite/g3doc/performance/models/official/mnist/dataset.py:65: The name tf.gfile.Exists is deprecated. Please use tf.io.gfile.exists instead.

W1001 10:57:29.508447 140568531904256 module_wrapper.py:137] From /tmpfs/src/temp/tensorflow/lite/g3doc/performance/models/official/mnist/dataset.py:65: The name tf.gfile.Exists is deprecated. Please use tf.io.gfile.exists instead.

WARNING:tensorflow:Entity <function dataset.<locals>.decode_image at 0x7fd8775c2c80> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: No module named 'tensorflow_estimator.contrib'
W1001 10:57:30.202761 140568531904256 ag_logging.py:146] Entity <function dataset.<locals>.decode_image at 0x7fd8775c2c80> could not be transformed and will be executed as-is. Please report this to the AutoGraph team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output. Cause: No module named 'tensorflow_estimator.contrib'
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
W1001 10:57:30.361464 140568531904256 deprecation.py:506] From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
INFO:tensorflow:Calling model_fn.
I1001 10:57:30.365632 140568531904256 estimator.py:1147] Calling model_fn.
WARNING:tensorflow:From models/official/mnist/mnist.py:127: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

W1001 10:57:30.492311 140568531904256 module_wrapper.py:137] From models/official/mnist/mnist.py:127: The name tf.train.AdamOptimizer is deprecated. Please use tf.compat.v1.train.AdamOptimizer instead.

WARNING:tensorflow:From models/official/mnist/mnist.py:130: The name tf.losses.sparse_softmax_cross_entropy is deprecated. Please use tf.compat.v1.losses.sparse_softmax_cross_entropy instead.

W1001 10:57:30.531999 140568531904256 module_wrapper.py:137] From models/official/mnist/mnist.py:130: The name tf.losses.sparse_softmax_cross_entropy is deprecated. Please use tf.compat.v1.losses.sparse_softmax_cross_entropy instead.

WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/python/ops/losses/losses_impl.py:121: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
W1001 10:57:30.542755 140568531904256 deprecation.py:323] From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/python/ops/losses/losses_impl.py:121: where (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
WARNING:tensorflow:From models/official/mnist/mnist.py:131: The name tf.metrics.accuracy is deprecated. Please use tf.compat.v1.metrics.accuracy instead.

W1001 10:57:30.552405 140568531904256 module_wrapper.py:137] From models/official/mnist/mnist.py:131: The name tf.metrics.accuracy is deprecated. Please use tf.compat.v1.metrics.accuracy instead.

WARNING:tensorflow:From models/official/mnist/mnist.py:140: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead.

W1001 10:57:30.577412 140568531904256 module_wrapper.py:137] From models/official/mnist/mnist.py:140: The name tf.summary.scalar is deprecated. Please use tf.compat.v1.summary.scalar instead.

INFO:tensorflow:Done calling model_fn.
I1001 10:57:30.798787 140568531904256 estimator.py:1149] Done calling model_fn.
INFO:tensorflow:Create CheckpointSaverHook.
I1001 10:57:30.836896 140568531904256 basic_session_run_hooks.py:541] Create CheckpointSaverHook.
INFO:tensorflow:Graph was finalized.
I1001 10:57:31.000674 140568531904256 monitored_session.py:240] Graph was finalized.
2019-10-01 10:57:31.001152: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-10-01 10:57:31.006621: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2200000000 Hz
2019-10-01 10:57:31.006960: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x66e9da0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2019-10-01 10:57:31.007002: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
INFO:tensorflow:Restoring parameters from /tmp/mnist_model/model.ckpt-1200
I1001 10:57:31.009893 140568531904256 saver.py:1284] Restoring parameters from /tmp/mnist_model/model.ckpt-1200
2019-10-01 10:57:31.022109: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
IteratorToStringHandle: CPU XLA_CPU 
IteratorV2: CPU XLA_CPU 
IteratorGetNext: CPU XLA_CPU 
MakeIterator: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  IteratorV2 (IteratorV2) /replica:0/task:0/device:GPU:0
  MakeIterator (MakeIterator) /replica:0/task:0/device:GPU:0
  IteratorToStringHandle (IteratorToStringHandle) /replica:0/task:0/device:GPU:0
  IteratorGetNext (IteratorGetNext) /replica:0/task:0/device:GPU:0

2019-10-01 10:57:31.022228: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
AssignAddVariableOp: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
Const: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  global_step/Initializer/zeros (Const) 
  global_step (VarHandleOp) /replica:0/task:0/device:GPU:0
  global_step/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  global_step/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  global_step/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Identity/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/AssignAddVariableOp (AssignAddVariableOp) /replica:0/task:0/device:GPU:0
  Adam/ReadVariableOp_4 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp (VarIsInitializedOp) 
  save/AssignVariableOp_26 (AssignVariableOp) /replica:0/task:0/device:GPU:0

2019-10-01 10:57:31.022407: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
ResourceApplyAdam: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
RandomUniform: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
Const: CPU XLA_CPU 
Mul: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 
Sub: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 
Add: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  conv2d/kernel/Initializer/random_uniform/shape (Const) 
  conv2d/kernel/Initializer/random_uniform/min (Const) 
  conv2d/kernel/Initializer/random_uniform/max (Const) 
  conv2d/kernel/Initializer/random_uniform/RandomUniform (RandomUniform) 
  conv2d/kernel/Initializer/random_uniform/sub (Sub) 
  conv2d/kernel/Initializer/random_uniform/mul (Mul) 
  conv2d/kernel/Initializer/random_uniform (Add) 
  conv2d/kernel (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/Conv2D/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  sequential/conv2d/Conv2D/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Adam/Initializer/zeros (Const) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Adam (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Adam/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Adam/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Adam/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Adam_1/Initializer/zeros (Const) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Adam_1 (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Adam_1/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Adam_1/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/kernel/Adam_1/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d/kernel/ResourceApplyAdam (ResourceApplyAdam) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_1 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_11 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_12 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp_1 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_11 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_12 (VarIsInitializedOp) 
  save/AssignVariableOp_5 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_6 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_7 (AssignVariableOp) /replica:0/task:0/device:GPU:0

2019-10-01 10:57:31.022736: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
Mul: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 
Const: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 
ResourceApplyAdam: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  conv2d/bias/Initializer/zeros (Const) 
  conv2d/bias (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/BiasAdd/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  sequential/conv2d/BiasAdd/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  beta1_power/Initializer/initial_value (Const) /replica:0/task:0/device:GPU:0
  beta1_power (VarHandleOp) /replica:0/task:0/device:GPU:0
  beta1_power/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  beta1_power/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  beta1_power/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  beta2_power/Initializer/initial_value (Const) /replica:0/task:0/device:GPU:0
  beta2_power (VarHandleOp) /replica:0/task:0/device:GPU:0
  beta2_power/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  beta2_power/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  beta2_power/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/Adam/Initializer/zeros (Const) /replica:0/task:0/device:GPU:0
  conv2d/bias/Adam (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/Adam/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/Adam/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/Adam/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/Adam_1/Initializer/zeros (Const) /replica:0/task:0/device:GPU:0
  conv2d/bias/Adam_1 (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/Adam_1/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/Adam_1/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d/bias/Adam_1/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d/kernel/ResourceApplyAdam/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d/kernel/ResourceApplyAdam/ReadVariableOp_1 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d/bias/ResourceApplyAdam/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d/bias/ResourceApplyAdam/ReadVariableOp_1 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d/bias/ResourceApplyAdam (ResourceApplyAdam) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d_1/kernel/ResourceApplyAdam/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d_1/kernel/ResourceApplyAdam/ReadVariableOp_1 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d_1/bias/ResourceApplyAdam/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d_1/bias/ResourceApplyAdam/ReadVariableOp_1 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense/kernel/ResourceApplyAdam/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense/kernel/ResourceApplyAdam/ReadVariableOp_1 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense/bias/ResourceApplyAdam/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense/bias/ResourceApplyAdam/ReadVariableOp_1 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense_1/kernel/ResourceApplyAdam/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense_1/kernel/ResourceApplyAdam/ReadVariableOp_1 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense_1/bias/ResourceApplyAdam/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense_1/bias/ResourceApplyAdam/ReadVariableOp_1 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/mul (Mul) /replica:0/task:0/device:GPU:0
  Adam/AssignVariableOp (AssignVariableOp) /replica:0/task:0/device:GPU:0
  Adam/ReadVariableOp_1 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/ReadVariableOp_2 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/mul_1 (Mul) /replica:0/task:0/device:GPU:0
  Adam/AssignVariableOp_1 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  Adam/ReadVariableOp_3 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_2 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_9 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_10 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_13 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_14 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp_2 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_9 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_10 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_13 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_14 (VarIsInitializedOp) 
  save/AssignVariableOp (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_1 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_2 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_3 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_4 (AssignVariableOp) /replica:0/task:0/device:GPU:0

2019-10-01 10:57:31.023120: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
ResourceApplyAdam: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
RandomUniform: CPU XLA_CPU 
Fill: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
Const: CPU XLA_CPU 
Mul: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 
Sub: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 
Add: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  conv2d_1/kernel/Initializer/random_uniform/shape (Const) 
  conv2d_1/kernel/Initializer/random_uniform/min (Const) 
  conv2d_1/kernel/Initializer/random_uniform/max (Const) 
  conv2d_1/kernel/Initializer/random_uniform/RandomUniform (RandomUniform) 
  conv2d_1/kernel/Initializer/random_uniform/sub (Sub) 
  conv2d_1/kernel/Initializer/random_uniform/mul (Mul) 
  conv2d_1/kernel/Initializer/random_uniform (Add) 
  conv2d_1/kernel (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/Conv2D/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  sequential/conv2d_1/Conv2D/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam/Initializer/zeros/shape_as_tensor (Const) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam/Initializer/zeros/Const (Const) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam/Initializer/zeros (Fill) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam_1/Initializer/zeros/shape_as_tensor (Const) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam_1/Initializer/zeros/Const (Const) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam_1/Initializer/zeros (Fill) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam_1 (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam_1/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam_1/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/kernel/Adam_1/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d_1/kernel/ResourceApplyAdam (ResourceApplyAdam) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_3 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_15 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_16 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp_3 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_15 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_16 (VarIsInitializedOp) 
  save/AssignVariableOp_11 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_12 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_13 (AssignVariableOp) /replica:0/task:0/device:GPU:0

2019-10-01 10:57:31.023348: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
ResourceApplyAdam: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
Const: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  conv2d_1/bias/Initializer/zeros (Const) 
  conv2d_1/bias (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/BiasAdd/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  sequential/conv2d_1/BiasAdd/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Adam/Initializer/zeros (Const) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Adam (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Adam/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Adam/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Adam/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Adam_1/Initializer/zeros (Const) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Adam_1 (VarHandleOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Adam_1/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Adam_1/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  conv2d_1/bias/Adam_1/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_conv2d_1/bias/ResourceApplyAdam (ResourceApplyAdam) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_4 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_17 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_18 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp_4 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_17 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_18 (VarIsInitializedOp) 
  save/AssignVariableOp_8 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_9 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_10 (AssignVariableOp) /replica:0/task:0/device:GPU:0

2019-10-01 10:57:31.023657: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
ResourceApplyAdam: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
RandomUniform: CPU XLA_CPU 
Fill: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
Const: CPU XLA_CPU 
Mul: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 
Sub: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 
Add: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  dense/kernel/Initializer/random_uniform/shape (Const) 
  dense/kernel/Initializer/random_uniform/min (Const) 
  dense/kernel/Initializer/random_uniform/max (Const) 
  dense/kernel/Initializer/random_uniform/RandomUniform (RandomUniform) 
  dense/kernel/Initializer/random_uniform/sub (Sub) 
  dense/kernel/Initializer/random_uniform/mul (Mul) 
  dense/kernel/Initializer/random_uniform (Add) 
  dense/kernel (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense/kernel/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense/kernel/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense/kernel/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense/MatMul/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  sequential/dense/MatMul/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam/Initializer/zeros/shape_as_tensor (Const) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam/Initializer/zeros/Const (Const) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam/Initializer/zeros (Fill) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam_1/Initializer/zeros/shape_as_tensor (Const) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam_1/Initializer/zeros/Const (Const) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam_1/Initializer/zeros (Fill) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam_1 (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam_1/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam_1/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense/kernel/Adam_1/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense/kernel/ResourceApplyAdam (ResourceApplyAdam) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_5 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_19 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_20 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp_5 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_19 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_20 (VarIsInitializedOp) 
  save/AssignVariableOp_17 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_18 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_19 (AssignVariableOp) /replica:0/task:0/device:GPU:0

2019-10-01 10:57:31.023994: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
ResourceApplyAdam: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 
Fill: CPU XLA_CPU 
Const: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  dense/bias/Initializer/zeros/shape_as_tensor (Const) 
  dense/bias/Initializer/zeros/Const (Const) 
  dense/bias/Initializer/zeros (Fill) 
  dense/bias (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense/bias/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense/bias/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense/bias/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense/BiasAdd/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  sequential/dense/BiasAdd/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense/bias/Adam/Initializer/zeros/shape_as_tensor (Const) /replica:0/task:0/device:GPU:0
  dense/bias/Adam/Initializer/zeros/Const (Const) /replica:0/task:0/device:GPU:0
  dense/bias/Adam/Initializer/zeros (Fill) /replica:0/task:0/device:GPU:0
  dense/bias/Adam (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense/bias/Adam/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense/bias/Adam/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense/bias/Adam/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense/bias/Adam_1/Initializer/zeros/shape_as_tensor (Const) /replica:0/task:0/device:GPU:0
  dense/bias/Adam_1/Initializer/zeros/Const (Const) /replica:0/task:0/device:GPU:0
  dense/bias/Adam_1/Initializer/zeros (Fill) /replica:0/task:0/device:GPU:0
  dense/bias/Adam_1 (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense/bias/Adam_1/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense/bias/Adam_1/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense/bias/Adam_1/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense/bias/ResourceApplyAdam (ResourceApplyAdam) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_6 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_21 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_22 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp_6 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_21 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_22 (VarIsInitializedOp) 
  save/AssignVariableOp_14 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_15 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_16 (AssignVariableOp) /replica:0/task:0/device:GPU:0

2019-10-01 10:57:31.024309: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
ResourceApplyAdam: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
RandomUniform: CPU XLA_CPU 
Fill: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
Const: CPU XLA_CPU 
Mul: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 
Sub: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 
Add: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  dense_1/kernel/Initializer/random_uniform/shape (Const) 
  dense_1/kernel/Initializer/random_uniform/min (Const) 
  dense_1/kernel/Initializer/random_uniform/max (Const) 
  dense_1/kernel/Initializer/random_uniform/RandomUniform (RandomUniform) 
  dense_1/kernel/Initializer/random_uniform/sub (Sub) 
  dense_1/kernel/Initializer/random_uniform/mul (Mul) 
  dense_1/kernel/Initializer/random_uniform (Add) 
  dense_1/kernel (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/MatMul/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  sequential/dense_1/MatMul/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam/Initializer/zeros/shape_as_tensor (Const) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam/Initializer/zeros/Const (Const) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam/Initializer/zeros (Fill) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam_1/Initializer/zeros/shape_as_tensor (Const) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam_1/Initializer/zeros/Const (Const) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam_1/Initializer/zeros (Fill) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam_1 (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam_1/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam_1/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/kernel/Adam_1/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense_1/kernel/ResourceApplyAdam (ResourceApplyAdam) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_7 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_23 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_24 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp_7 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_23 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_24 (VarIsInitializedOp) 
  save/AssignVariableOp_23 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_24 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_25 (AssignVariableOp) /replica:0/task:0/device:GPU:0

2019-10-01 10:57:31.024575: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
ResourceApplyAdam: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
Const: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  dense_1/bias/Initializer/zeros (Const) 
  dense_1/bias (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/BiasAdd/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  sequential/dense_1/BiasAdd/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/Adam/Initializer/zeros (Const) /replica:0/task:0/device:GPU:0
  dense_1/bias/Adam (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/Adam/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/Adam/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/Adam/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/Adam_1/Initializer/zeros (Const) /replica:0/task:0/device:GPU:0
  dense_1/bias/Adam_1 (VarHandleOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/Adam_1/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/Adam_1/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  dense_1/bias/Adam_1/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  Adam/update_dense_1/bias/ResourceApplyAdam (ResourceApplyAdam) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_8 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_25 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables/VarIsInitializedOp_26 (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp_8 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_25 (VarIsInitializedOp) 
  report_uninitialized_variables_1/VarIsInitializedOp_26 (VarIsInitializedOp) 
  save/AssignVariableOp_20 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_21 (AssignVariableOp) /replica:0/task:0/device:GPU:0
  save/AssignVariableOp_22 (AssignVariableOp) /replica:0/task:0/device:GPU:0

2019-10-01 10:57:31.024910: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
AssignAddVariableOp: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
Const: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  accuracy/total/Initializer/zeros (Const) 
  accuracy/total (VarHandleOp) /replica:0/task:0/device:GPU:0
  accuracy/total/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  accuracy/total/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  accuracy/total/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  accuracy/AssignAddVariableOp (AssignAddVariableOp) /replica:0/task:0/device:GPU:0
  accuracy/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  accuracy/value/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  accuracy/update_op/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp_27 (VarIsInitializedOp) 

2019-10-01 10:57:31.025033: W tensorflow/core/common_runtime/colocation_graph.cc:983] Failed to place the graph without changing the devices of some resources. Some of the operations (that had to be colocated with resource generating operations) are not supported on the resources' devices. Current candidate devices are [
  /job:localhost/replica:0/task:0/device:CPU:0
  /job:localhost/replica:0/task:0/device:XLA_CPU:0].
See below for details of this colocation group:
Colocation Debug Info:
Colocation group had the following types and supported devices: 
Root Member(assigned_device_name_index_=-1 requested_device_name_='/replica:0/task:0/device:GPU:0' assigned_device_name_='' resource_device_name_='/replica:0/task:0/device:GPU:0' supported_device_types_=[CPU, XLA_CPU] possible_devices_=[]
AssignAddVariableOp: CPU XLA_CPU 
ReadVariableOp: CPU XLA_CPU 
AssignVariableOp: CPU XLA_CPU 
VarIsInitializedOp: CPU XLA_CPU 
Const: CPU XLA_CPU 
VarHandleOp: CPU XLA_CPU 

Colocation members, user-requested devices, and framework assigned devices, if any:
  accuracy/count/Initializer/zeros (Const) 
  accuracy/count (VarHandleOp) /replica:0/task:0/device:GPU:0
  accuracy/count/IsInitialized/VarIsInitializedOp (VarIsInitializedOp) /replica:0/task:0/device:GPU:0
  accuracy/count/Assign (AssignVariableOp) /replica:0/task:0/device:GPU:0
  accuracy/count/Read/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  accuracy/AssignAddVariableOp_1 (AssignAddVariableOp) /replica:0/task:0/device:GPU:0
  accuracy/ReadVariableOp_1 (ReadVariableOp) /replica:0/task:0/device:GPU:0
  accuracy/Maximum/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  accuracy/Maximum_1/ReadVariableOp (ReadVariableOp) /replica:0/task:0/device:GPU:0
  report_uninitialized_variables_1/VarIsInitializedOp_28 (VarIsInitializedOp) 

WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/python/training/saver.py:1069: get_checkpoint_mtimes (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file utilities to get mtimes.
W1001 10:57:31.087836 140568531904256 deprecation.py:323] From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/python/training/saver.py:1069: get_checkpoint_mtimes (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file utilities to get mtimes.
INFO:tensorflow:Running local_init_op.
I1001 10:57:31.124178 140568531904256 session_manager.py:500] Running local_init_op.
INFO:tensorflow:Done running local_init_op.
I1001 10:57:31.140701 140568531904256 session_manager.py:502] Done running local_init_op.
INFO:tensorflow:Saving checkpoints for 1200 into /tmp/mnist_model/model.ckpt.
I1001 10:57:31.468534 140568531904256 basic_session_run_hooks.py:606] Saving checkpoints for 1200 into /tmp/mnist_model/model.ckpt.
INFO:tensorflow:cross_entropy = 0.037972555, learning_rate = 1e-04, train_accuracy = 0.99
I1001 10:57:37.335710 140568531904256 basic_session_run_hooks.py:262] cross_entropy = 0.037972555, learning_rate = 1e-04, train_accuracy = 0.99
INFO:tensorflow:loss = 0.037972555, step = 1200
I1001 10:57:37.336319 140568531904256 basic_session_run_hooks.py:262] loss = 0.037972555, step = 1200
INFO:tensorflow:global_step/sec: 5.92771
I1001 10:57:54.205220 140568531904256 basic_session_run_hooks.py:692] global_step/sec: 5.92771
INFO:tensorflow:cross_entropy = 0.115275815, learning_rate = 1e-04, train_accuracy = 0.975 (16.870 sec)
I1001 10:57:54.206113 140568531904256 basic_session_run_hooks.py:260] cross_entropy = 0.115275815, learning_rate = 1e-04, train_accuracy = 0.975 (16.870 sec)
INFO:tensorflow:loss = 0.115275815, step = 1300 (16.870 sec)
I1001 10:57:54.206480 140568531904256 basic_session_run_hooks.py:260] loss = 0.115275815, step = 1300 (16.870 sec)
INFO:tensorflow:global_step/sec: 6.08753
I1001 10:58:10.632254 140568531904256 basic_session_run_hooks.py:692] global_step/sec: 6.08753
INFO:tensorflow:cross_entropy = 0.08243354, learning_rate = 1e-04, train_accuracy = 0.97 (16.427 sec)
I1001 10:58:10.633177 140568531904256 basic_session_run_hooks.py:260] cross_entropy = 0.08243354, learning_rate = 1e-04, train_accuracy = 0.97 (16.427 sec)
INFO:tensorflow:loss = 0.08243354, step = 1400 (16.427 sec)
I1001 10:58:10.633478 140568531904256 basic_session_run_hooks.py:260] loss = 0.08243354, step = 1400 (16.427 sec)
INFO:tensorflow:global_step/sec: 6.18753
I1001 10:58:26.793767 140568531904256 basic_session_run_hooks.py:692] global_step/sec: 6.18753
INFO:tensorflow:cross_entropy = 0.34156707, learning_rate = 1e-04, train_accuracy = 0.965 (16.162 sec)
I1001 10:58:26.794686 140568531904256 basic_session_run_hooks.py:260] cross_entropy = 0.34156707, learning_rate = 1e-04, train_accuracy = 0.965 (16.162 sec)
INFO:tensorflow:loss = 0.34156707, step = 1500 (16.161 sec)
I1001 10:58:26.794941 140568531904256 basic_session_run_hooks.py:260] loss = 0.34156707, step = 1500 (16.161 sec)
INFO:tensorflow:global_step/sec: 6.21339
I1001 10:58:42.888058 140568531904256 basic_session_run_hooks.py:692] global_step/sec: 6.21339
INFO:tensorflow:cross_entropy = 0.13578896, learning_rate = 1e-04, train_accuracy = 0.964 (16.094 sec)
I1001 10:58:42.889008 140568531904256 basic_session_run_hooks.py:260] cross_entropy = 0.13578896, learning_rate = 1e-04, train_accuracy = 0.964 (16.094 sec)
INFO:tensorflow:loss = 0.13578896, step = 1600 (16.094 sec)
I1001 10:58:42.889267 140568531904256 basic_session_run_hooks.py:260] loss = 0.13578896, step = 1600 (16.094 sec)
INFO:tensorflow:global_step/sec: 6.22984
I1001 10:58:58.939831 140568531904256 basic_session_run_hooks.py:692] global_step/sec: 6.22984
INFO:tensorflow:cross_entropy = 0.037469476, learning_rate = 1e-04, train_accuracy = 0.9683333 (16.052 sec)
I1001 10:58:58.940741 140568531904256 basic_session_run_hooks.py:260] cross_entropy = 0.037469476, learning_rate = 1e-04, train_accuracy = 0.9683333 (16.052 sec)
INFO:tensorflow:loss = 0.037469476, step = 1700 (16.052 sec)
I1001 10:58:58.941108 140568531904256 basic_session_run_hooks.py:260] loss = 0.037469476, step = 1700 (16.052 sec)
INFO:tensorflow:Saving checkpoints for 1800 into /tmp/mnist_model/model.ckpt.
I1001 10:59:14.856639 140568531904256 basic_session_run_hooks.py:606] Saving checkpoints for 1800 into /tmp/mnist_model/model.ckpt.
INFO:tensorflow:Loss for final step: 0.03278794.
I1001 10:59:15.002225 140568531904256 estimator.py:371] Loss for final step: 0.03278794.
WARNING:tensorflow:From models/official/mnist/mnist.py:210: DatasetV1.make_one_shot_iterator (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `for ... in dataset:` to iterate over a dataset. If using `tf.estimator`, return the `Dataset` object directly from your input function. As a last resort, you can use `tf.compat.v1.data.make_one_shot_iterator(dataset)`.
W1001 10:59:15.128636 140568531904256 deprecation.py:323] From models/official/mnist/mnist.py:210: DatasetV1.make_one_shot_iterator (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `for ... in dataset:` to iterate over a dataset. If using `tf.estimator`, return the `Dataset` object directly from your input function. As a last resort, you can use `tf.compat.v1.data.make_one_shot_iterator(dataset)`.
INFO:tensorflow:Calling model_fn.
I1001 10:59:15.144644 140568531904256 estimator.py:1147] Calling model_fn.
INFO:tensorflow:Done calling model_fn.
I1001 10:59:15.326215 140568531904256 estimator.py:1149] Done calling model_fn.
INFO:tensorflow:Starting evaluation at 2019-10-01T10:59:15Z
I1001 10:59:15.346354 140568531904256 evaluation.py:255] Starting evaluation at 2019-10-01T10:59:15Z
INFO:tensorflow:Graph was finalized.
I1001 10:59:15.427899 140568531904256 monitored_session.py:240] Graph was finalized.
INFO:tensorflow:Restoring parameters from /tmp/mnist_model/model.ckpt-1800
I1001 10:59:15.429352 140568531904256 saver.py:1284] Restoring parameters from /tmp/mnist_model/model.ckpt-1800
INFO:tensorflow:Running local_init_op.
I1001 10:59:15.479628 140568531904256 session_manager.py:500] Running local_init_op.
INFO:tensorflow:Done running local_init_op.
I1001 10:59:15.492940 140568531904256 session_manager.py:502] Done running local_init_op.
INFO:tensorflow:Finished evaluation at 2019-10-01-10:59:20
I1001 10:59:20.816722 140568531904256 evaluation.py:275] Finished evaluation at 2019-10-01-10:59:20
INFO:tensorflow:Saving dict for global step 1800: accuracy = 0.9854, global_step = 1800, loss = 0.046254545
I1001 10:59:20.817019 140568531904256 estimator.py:2049] Saving dict for global step 1800: accuracy = 0.9854, global_step = 1800, loss = 0.046254545
INFO:tensorflow:Saving 'checkpoint_path' summary for global step 1800: /tmp/mnist_model/model.ckpt-1800
I1001 10:59:20.869001 140568531904256 estimator.py:2109] Saving 'checkpoint_path' summary for global step 1800: /tmp/mnist_model/model.ckpt-1800

Evaluation results:
    {'loss': 0.046254545, 'accuracy': 0.9854, 'global_step': 1800}

WARNING:tensorflow:From models/official/mnist/mnist.py:234: Estimator.export_savedmodel (from tensorflow_estimator.python.estimator.estimator) is deprecated and will be removed in a future version.
Instructions for updating:
This function has been renamed, use `export_saved_model` instead.
W1001 10:59:20.870564 140568531904256 deprecation.py:323] From models/official/mnist/mnist.py:234: Estimator.export_savedmodel (from tensorflow_estimator.python.estimator.estimator) is deprecated and will be removed in a future version.
Instructions for updating:
This function has been renamed, use `export_saved_model` instead.
INFO:tensorflow:Calling model_fn.
I1001 10:59:20.877354 140568531904256 estimator.py:1147] Calling model_fn.
INFO:tensorflow:Done calling model_fn.
I1001 10:59:21.017292 140568531904256 estimator.py:1149] Done calling model_fn.
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/python/saved_model/signature_def_utils_impl.py:201: build_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.utils.build_tensor_info or tf.compat.v1.saved_model.build_tensor_info.
W1001 10:59:21.017674 140568531904256 deprecation.py:323] From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/python/saved_model/signature_def_utils_impl.py:201: build_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.utils.build_tensor_info or tf.compat.v1.saved_model.build_tensor_info.
INFO:tensorflow:Signatures INCLUDED in export for Predict: ['classify', 'serving_default']
I1001 10:59:21.018191 140568531904256 export_utils.py:170] Signatures INCLUDED in export for Predict: ['classify', 'serving_default']
INFO:tensorflow:Signatures INCLUDED in export for Eval: None
I1001 10:59:21.018368 140568531904256 export_utils.py:170] Signatures INCLUDED in export for Eval: None
INFO:tensorflow:Signatures INCLUDED in export for Train: None
I1001 10:59:21.018509 140568531904256 export_utils.py:170] Signatures INCLUDED in export for Train: None
INFO:tensorflow:Signatures INCLUDED in export for Classify: None
I1001 10:59:21.018650 140568531904256 export_utils.py:170] Signatures INCLUDED in export for Classify: None
INFO:tensorflow:Signatures INCLUDED in export for Regress: None
I1001 10:59:21.018793 140568531904256 export_utils.py:170] Signatures INCLUDED in export for Regress: None
INFO:tensorflow:Restoring parameters from /tmp/mnist_model/model.ckpt-1800
I1001 10:59:21.043310 140568531904256 saver.py:1284] Restoring parameters from /tmp/mnist_model/model.ckpt-1800
INFO:tensorflow:Assets added to graph.
I1001 10:59:21.068533 140568531904256 builder_impl.py:665] Assets added to graph.
INFO:tensorflow:No assets to write.
I1001 10:59:21.068777 140568531904256 builder_impl.py:460] No assets to write.
INFO:tensorflow:SavedModel written to: /tmp/mnist_saved_model/temp-b'1569952760'/saved_model.pb
I1001 10:59:21.125460 140568531904256 builder_impl.py:425] SavedModel written to: /tmp/mnist_saved_model/temp-b'1569952760'/saved_model.pb

This training won't take long because you're training the model for just a single epoch, which trains to about 96% accuracy.

Convert to a TensorFlow Lite model

Using the Python TFLiteConverter, you can now convert the trained model into a TensorFlow Lite model.

The trained model is saved in the saved_models_root directory, which is named with a timestamp. So select the most recent directory:

saved_model_dir = str(sorted(pathlib.Path(saved_models_root).glob("*"))[-1])
saved_model_dir
'/tmp/mnist_saved_model/1569952760'

Now load the model using the TFLiteConverter:

import tensorflow as tf
tf.enable_eager_execution()
tf.logging.set_verbosity(tf.logging.DEBUG)

converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
tflite_model = converter.convert()
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/lite/python/convert_saved_model.py:60: load (from tensorflow.python.saved_model.loader_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.loader.load or tf.compat.v1.saved_model.load. There will be a new function for importing SavedModels in Tensorflow 2.0.
INFO:tensorflow:Restoring parameters from /tmp/mnist_saved_model/1569952760/variables/variables
INFO:tensorflow:The given SavedModel MetaGraphDef contains SignatureDefs with the following keys: {'classify', 'serving_default'}
INFO:tensorflow:input tensors info: 
INFO:tensorflow:Tensor's key in saved_model's tensor_map: image
INFO:tensorflow: tensor name: Placeholder:0, shape: (-1, 28, 28), type: DT_FLOAT
INFO:tensorflow:output tensors info: 
INFO:tensorflow:Tensor's key in saved_model's tensor_map: probabilities
INFO:tensorflow: tensor name: Softmax:0, shape: (-1, 10), type: DT_FLOAT
INFO:tensorflow:Tensor's key in saved_model's tensor_map: classes
INFO:tensorflow: tensor name: ArgMax:0, shape: (-1), type: DT_INT64
INFO:tensorflow:Restoring parameters from /tmp/mnist_saved_model/1569952760/variables/variables
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/lite/python/util.py:249: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.convert_variables_to_constants`
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.5/site-packages/tensorflow_core/python/framework/graph_util_impl.py:277: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.compat.v1.graph_util.extract_sub_graph`
INFO:tensorflow:Froze 8 variables.
INFO:tensorflow:Converted 8 variables to const ops.

Write it out to a .tflite file:

tflite_models_dir = pathlib.Path("/tmp/mnist_tflite_models/")
tflite_models_dir.mkdir(exist_ok=True, parents=True)
tflite_model_file = tflite_models_dir/"mnist_model.tflite"
tflite_model_file.write_bytes(tflite_model)
13101352

Now you have a trained MNIST model that's converted to a .tflite file, but it's still using 32-bit float values for all parameter data.

So let's convert the model again, this time using quantization...

Convert using quantization

First, first set the optimizations flag to optimize for size:

tf.logging.set_verbosity(tf.logging.INFO)
converter.optimizations = [tf.lite.Optimize.DEFAULT]

Now, in order to create quantized values with an accurate dynamic range of activations, you need to provide a representative dataset:

mnist_train, _ = tf.keras.datasets.mnist.load_data()
images = tf.cast(mnist_train[0], tf.float32)/255.0
mnist_ds = tf.data.Dataset.from_tensor_slices((images)).batch(1)
def representative_data_gen():
  for input_value in mnist_ds.take(100):
    yield [input_value]

converter.representative_dataset = representative_data_gen

Finally, convert the model to TensorFlow Lite format:

tflite_model_quant = converter.convert()
tflite_model_quant_file = tflite_models_dir/"mnist_model_quant.tflite"
tflite_model_quant_file.write_bytes(tflite_model_quant)
3284088

Note how the resulting file is approximately 1/4 the size:

!ls -lh {tflite_models_dir}
total 22M
-rw-rw-r-- 1 kbuilder kbuilder 6.3M Oct  1 10:56 mnist_model_quant_f16.tflite
-rw-rw-r-- 1 kbuilder kbuilder 3.2M Oct  1 10:59 mnist_model_quant.tflite
-rw-rw-r-- 1 kbuilder kbuilder  13M Oct  1 10:59 mnist_model.tflite

Your model should now be fully quantized. However, if you convert a model that includes any operations that TensorFlow Lite cannot quantize, those ops are left in floating point. This allows for conversion to complete so you have a smaller and more efficient model, but the model won't be compatible with some ML accelerators that require full integer quantization. Also, by default, the converted model still use float input and outputs, which also is not compatible with some accelerators.

So to ensure that the converted model is fully quantized (make the converter throw an error if it encounters an operation it cannot quantize), and to use integers for the model's input and output, you need to convert the model again using these additional configurations:

converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8

tflite_model_quant = converter.convert()
tflite_model_quant_file = tflite_models_dir/"mnist_model_quant_io.tflite"
tflite_model_quant_file.write_bytes(tflite_model_quant)
3284160

In this example, the resulting model size remains the same because all operations successfully quantized to begin with. However, this new model now uses quantized input and output, making it compatible with more accelerators, such as the Coral Edge TPU.

In the following sections, notice that we are now handling two TensorFlow Lite models: tflite_model_file is the converted model that still uses floating-point parameters, and tflite_model_quant_file is the same model converted with full integer quantization, including uint8 input and output.

Run the TensorFlow Lite models

Run the TensorFlow Lite model using the Python TensorFlow Lite Interpreter.

Load the test data

First, let's load the MNIST test data to feed to the model. Because the quantized model expects uint8 input data, we need to create a separate dataset for that model:

import numpy as np
_, mnist_test = tf.keras.datasets.mnist.load_data()
labels = mnist_test[1]

# Load data for float model
images = tf.cast(mnist_test[0], tf.float32)/255.0
mnist_ds = tf.data.Dataset.from_tensor_slices((images, labels)).batch(1)

# Load data for quantized model
images_uint8 = tf.cast(mnist_test[0], tf.uint8)
mnist_ds_uint8 = tf.data.Dataset.from_tensor_slices((images_uint8, labels)).batch(1)

Load the model into the interpreters

interpreter = tf.lite.Interpreter(model_path=str(tflite_model_file))
interpreter.allocate_tensors()
interpreter_quant = tf.lite.Interpreter(model_path=str(tflite_model_quant_file))
interpreter_quant.allocate_tensors()

Test the models on one image

First test it on the float model:

for img, label in mnist_ds:
  break

interpreter.set_tensor(interpreter.get_input_details()[0]["index"], img)
interpreter.invoke()
predictions = interpreter.get_tensor(
    interpreter.get_output_details()[0]["index"])
import matplotlib.pylab as plt

plt.imshow(img[0])
template = "True:{true}, predicted:{predict}"
_ = plt.title(template.format(true= str(label[0].numpy()),
                              predict=str(predictions[0])))
plt.grid(False)

Now test the quantized model (using the uint8 data):

for img, label in mnist_ds_uint8:
  break

interpreter_quant.set_tensor(
    interpreter_quant.get_input_details()[0]["index"], img)
interpreter_quant.invoke()
predictions = interpreter_quant.get_tensor(
    interpreter_quant.get_output_details()[0]["index"])
plt.imshow(img[0])
template = "True:{true}, predicted:{predict}"
_ = plt.title(template.format(true= str(label[0].numpy()),
                              predict=str(predictions[0])))
plt.grid(False)

png

Evaluate the models

def eval_model(interpreter, mnist_ds):
  total_seen = 0
  num_correct = 0

  input_index = interpreter.get_input_details()[0]["index"]
  output_index = interpreter.get_output_details()[0]["index"]

  for img, label in mnist_ds:
    total_seen += 1
    interpreter.set_tensor(input_index, img)
    interpreter.invoke()
    predictions = interpreter.get_tensor(output_index)
    if predictions == label.numpy():
      num_correct += 1

    if total_seen % 500 == 0:
      print("Accuracy after %i images: %f" %
            (total_seen, float(num_correct) / float(total_seen)))

  return float(num_correct) / float(total_seen)
# Create smaller dataset for demonstration purposes
mnist_ds_demo = mnist_ds.take(2000)

print(eval_model(interpreter, mnist_ds_demo))
Accuracy after 500 images: 0.988000
Accuracy after 1000 images: 0.986000
Accuracy after 1500 images: 0.980667
Accuracy after 2000 images: 0.981000
0.981

Repeat the evaluation on the fully quantized model using the uint8 data:

# NOTE: Colab runs on server CPUs, and TensorFlow Lite currently
# doesn't have super optimized server CPU kernels. So this part may be
# slower than the above float interpreter. But for mobile CPUs, considerable
# speedup can be observed.
mnist_ds_demo_uint8 = mnist_ds_uint8.take(2000)

print(eval_model(interpreter_quant, mnist_ds_demo_uint8))
Accuracy after 500 images: 0.988000
Accuracy after 1000 images: 0.986000
Accuracy after 1500 images: 0.980667
Accuracy after 2000 images: 0.981500
0.9815

In this example, you have fully quantized a model with almost no difference in the accuracy, compared to the above float model.