 View on TensorFlow.org
|
 Run in Google Colab
|
 View source on GitHub
|
 Download notebook
|
Overview
This tutorial demonstrates how you can save and load models in a SavedModel format with tf.distribute.Strategy during or after training. There are two kinds of APIs for saving and loading a Keras model: high-level (tf.keras.Model.save and tf.keras.models.load_model) and low-level (tf.saved_model.save and tf.saved_model.load).
To learn about SavedModel and serialization in general, please read the saved model guide, and the Keras model serialization guide. Let's start with a simple example.
Import dependencies:
import tensorflow_datasets as tfds
import tensorflow as tf
Load and prepare the data with TensorFlow Datasets and tf.data, and create the model using tf.distribute.MirroredStrategy:
mirrored_strategy = tf.distribute.MirroredStrategy()
def get_data():
datasets = tfds.load(name='mnist', as_supervised=True)
mnist_train, mnist_test = datasets['train'], datasets['test']
BUFFER_SIZE = 10000
BATCH_SIZE_PER_REPLICA = 64
BATCH_SIZE = BATCH_SIZE_PER_REPLICA * mirrored_strategy.num_replicas_in_sync
def scale(image, label):
image = tf.cast(image, tf.float32)
image /= 255
return image, label
train_dataset = mnist_train.map(scale).cache().shuffle(BUFFER_SIZE).batch(BATCH_SIZE)
eval_dataset = mnist_test.map(scale).batch(BATCH_SIZE)
return train_dataset, eval_dataset
def get_model():
with mirrored_strategy.scope():
model = tf.keras.Sequential([
tf.keras.layers.Conv2D(32, 3, activation='relu', input_shape=(28, 28, 1)),
tf.keras.layers.MaxPooling2D(),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(64, activation='relu'),
tf.keras.layers.Dense(10)
])
model.compile(loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=tf.keras.optimizers.Adam(),
metrics=[tf.metrics.SparseCategoricalAccuracy()])
return model
INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0', '/job:localhost/replica:0/task:0/device:GPU:1', '/job:localhost/replica:0/task:0/device:GPU:2', '/job:localhost/replica:0/task:0/device:GPU:3')
Train the model with tf.keras.Model.fit:
model = get_model()
train_dataset, eval_dataset = get_data()
model.fit(train_dataset, epochs=2)
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
W0000 00:00:1793849632.790721 72427 auto_shard.cc:559] The `assert_cardinality` transformation is currently not handled by the auto-shard rewrite and will be removed.
Epoch 1/2
INFO:tensorflow:Collective all_reduce tensors: 6 all_reduces, num_devices = 4, group_size = 4, implementation = CommunicationImplementation.NCCL, num_packs = 1
INFO:tensorflow:Collective all_reduce tensors: 6 all_reduces, num_devices = 4, group_size = 4, implementation = CommunicationImplementation.NCCL, num_packs = 1
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Collective all_reduce tensors: 6 all_reduces, num_devices = 4, group_size = 4, implementation = CommunicationImplementation.NCCL, num_packs = 1
INFO:tensorflow:Collective all_reduce tensors: 6 all_reduces, num_devices = 4, group_size = 4, implementation = CommunicationImplementation.NCCL, num_packs = 1
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
235/235 [==============================] - ETA: 0s - loss: 0.3512 - sparse_categorical_accuracy: 0.9068
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
INFO:tensorflow:Reduce to /job:localhost/replica:0/task:0/device:CPU:0 then broadcast to ('/job:localhost/replica:0/task:0/device:CPU:0',).
235/235 [==============================] - 7s 8ms/step - loss: 0.3512 - sparse_categorical_accuracy: 0.9068
Epoch 2/2
235/235 [==============================] - 2s 7ms/step - loss: 0.0980 - sparse_categorical_accuracy: 0.9718
<tf_keras.src.callbacks.History at 0x7f8e8c5d1090>
Save and load the model
Now that you have a simple model to work with, let's explore the saving/loading APIs. There are two kinds of APIs available:
- High-level (Keras):
Model.saveandtf.keras.models.load_model(.keraszip archive format) - Low-level:
tf.saved_model.saveandtf.saved_model.load(TF SavedModel format)
The Keras API
Here is an example of saving and loading a model with the Keras API:
keras_model_path = '/tmp/keras_save.keras'
model.save(keras_model_path)
Restore the model without tf.distribute.Strategy:
restored_keras_model = tf.keras.models.load_model(keras_model_path)
restored_keras_model.fit(train_dataset, epochs=2)
Epoch 1/2 235/235 [==============================] - 2s 4ms/step - loss: 0.0663 - sparse_categorical_accuracy: 0.9811 Epoch 2/2 235/235 [==============================] - 1s 4ms/step - loss: 0.0524 - sparse_categorical_accuracy: 0.9851 <tf_keras.src.callbacks.History at 0x7f8e8c4ebcd0>
After restoring the model, you can continue training on it, even without needing to call Model.compile again, since it was already compiled before saving. The model is saved a Keras zip archive format, marked by the .keras extension. For more information, please refer to the guide on Keras saving.
Now, restore the model and train it using a tf.distribute.Strategy:
another_strategy = tf.distribute.OneDeviceStrategy('/cpu:0')
with another_strategy.scope():
restored_keras_model_ds = tf.keras.models.load_model(keras_model_path)
restored_keras_model_ds.fit(train_dataset, epochs=2)
Epoch 1/2 W0000 00:00:1793849645.755511 72427 auto_shard.cc:559] The `assert_cardinality` transformation is currently not handled by the auto-shard rewrite and will be removed. W0000 00:00:1793849645.810021 72427 dataset.cc:997] Input of GeneratorDatasetOp::Dataset will not be optimized because the dataset does not implement the AsGraphDefInternal() method needed to apply optimizations. W0000 00:00:1793849646.355984 72612 platform_util.cc:146] Allowed device set contains 4 devices, but platform only sees 1 235/235 [==============================] - 6s 22ms/step - loss: 0.0669 - sparse_categorical_accuracy: 0.9812 Epoch 2/2 235/235 [==============================] - 5s 22ms/step - loss: 0.0518 - sparse_categorical_accuracy: 0.9854
As the Model.fit output shows, loading works as expected with tf.distribute.Strategy. The strategy used here does not have to be the same strategy used before saving.
The tf.saved_model API
Saving the model with lower-level API is similar to the Keras API:
model = get_model() # get a fresh model
saved_model_path = '/tmp/tf_save'
tf.saved_model.save(model, saved_model_path)
INFO:tensorflow:Assets written to: /tmp/tf_save/assets INFO:tensorflow:Assets written to: /tmp/tf_save/assets
Loading can be done with tf.saved_model.load. However, since it is a lower-level API (and hence has a wider range of use cases), it does not return a Keras model. Instead, it returns an object that contain functions that can be used to do inference. For example:
DEFAULT_FUNCTION_KEY = 'serving_default'
loaded = tf.saved_model.load(saved_model_path)
inference_func = loaded.signatures[DEFAULT_FUNCTION_KEY]
The loaded object may contain multiple functions, each associated with a key. The "serving_default" key is the default key for the inference function with a saved Keras model. To do inference with this function:
predict_dataset = eval_dataset.map(lambda image, label: image)
for batch in predict_dataset.take(1):
print(inference_func(batch))
{'dense_3': <tf.Tensor: shape=(256, 10), dtype=float32, numpy=
array([[ 0.01363315, 0.00740635, 0.06883518, ..., -0.15935549,
-0.03547644, -0.08697496],
[-0.20836565, -0.11242262, 0.03826099, ..., -0.19828624,
-0.02436422, 0.01445359],
[-0.287029 , -0.0008188 , 0.05262598, ..., -0.18332972,
-0.17994417, 0.03955157],
...,
[-0.02865289, -0.0472459 , 0.03511006, ..., -0.05890357,
-0.00484711, 0.04592701],
[-0.17038591, 0.04257239, 0.09329042, ..., 0.00182852,
-0.0877681 , 0.06233243],
[-0.13186827, -0.03954301, -0.01671081, ..., -0.19457608,
0.06994207, 0.05223262]], shape=(256, 10), dtype=float32)>}
W0000 00:00:1793849658.061363 74491 cache_dataset_ops.cc:912] The calling iterator did not fully read the dataset being cached. In order to avoid unexpected truncation of the dataset, the partially cached contents of the dataset will be discarded. This can happen if you have an input pipeline similar to `dataset.cache().take(k).repeat()`. You should use `dataset.take(k).cache().repeat()` instead.
You can also load and do inference in a distributed manner:
another_strategy = tf.distribute.MirroredStrategy()
with another_strategy.scope():
loaded = tf.saved_model.load(saved_model_path)
inference_func = loaded.signatures[DEFAULT_FUNCTION_KEY]
dist_predict_dataset = another_strategy.experimental_distribute_dataset(
predict_dataset)
# Calling the function in a distributed manner
for batch in dist_predict_dataset:
result = another_strategy.run(inference_func, args=(batch,))
print(result)
break
INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0', '/job:localhost/replica:0/task:0/device:GPU:1', '/job:localhost/replica:0/task:0/device:GPU:2', '/job:localhost/replica:0/task:0/device:GPU:3')
INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0', '/job:localhost/replica:0/task:0/device:GPU:1', '/job:localhost/replica:0/task:0/device:GPU:2', '/job:localhost/replica:0/task:0/device:GPU:3')
W0000 00:00:1793849658.271626 72427 auto_shard.cc:559] The `assert_cardinality` transformation is currently not handled by the auto-shard rewrite and will be removed.
WARNING:tensorflow:Using MirroredStrategy eagerly has significant overhead currently. We will be working on improving this in the future, but for now please wrap `call_for_each_replica` or `experimental_run` or `run` inside a tf.function to get the best performance.
WARNING:tensorflow:Using MirroredStrategy eagerly has significant overhead currently. We will be working on improving this in the future, but for now please wrap `call_for_each_replica` or `experimental_run` or `run` inside a tf.function to get the best performance.
{'dense_3': PerReplica:{
0: <tf.Tensor: shape=(64, 10), dtype=float32, numpy=
array([[ 0.01363315, 0.00740636, 0.06883521, -0.04706157, 0.01466455,
-0.05795628, 0.02906879, -0.15935549, -0.03547646, -0.08697496],
[-0.20836563, -0.11242266, 0.038261 , 0.12509897, -0.19162247,
-0.06512845, -0.08218981, -0.19828624, -0.0243642 , 0.01445359],
[-0.287029 , -0.00081877, 0.05262598, 0.02525267, -0.18970576,
0.04488207, 0.07318412, -0.18332973, -0.17994413, 0.03955162],
[-0.14383969, 0.00919317, 0.07536907, 0.0130841 , 0.01103027,
0.00919377, 0.07835818, -0.03554387, -0.03758406, 0.07509081],
[-0.12187575, -0.00188494, 0.01004267, 0.02682964, -0.01129058,
0.01043785, 0.07141665, -0.04528458, -0.03394434, 0.13102454],
[-0.13578512, -0.04297263, -0.08502331, -0.06782605, -0.20060748,
-0.09838212, 0.00203285, -0.2088669 , 0.04390345, 0.05129954],
[-0.14505367, -0.15662688, 0.00285584, 0.03374016, -0.16670701,
-0.00555189, -0.09607405, -0.1662525 , 0.00166226, 0.06701398],
[-0.12339575, 0.0375549 , -0.01199043, -0.01323093, -0.08028521,
-0.04190829, 0.05663068, -0.11464871, 0.04137245, 0.01705423],
[-0.16785643, 0.03847981, 0.03197426, -0.08972374, -0.15736905,
0.02656541, 0.03354099, -0.1473562 , 0.04561754, 0.03976174],
[-0.09289599, -0.01074573, 0.07284805, -0.00061153, -0.10590565,
-0.02732992, 0.07809663, 0.00675575, 0.08359846, 0.00837478],
[-0.09096272, -0.00712202, -0.01378823, -0.04298315, -0.04193742,
-0.06040108, 0.02124261, -0.14272219, 0.00356204, 0.03260209],
[-0.12396859, -0.02900437, 0.03311907, -0.01594931, -0.16163579,
-0.0409958 , 0.05562737, -0.18754408, 0.08690193, 0.00312155],
[-0.05523062, -0.0098388 , 0.09710416, -0.02637886, -0.04228452,
0.11208559, 0.04183143, -0.08237582, -0.10622749, 0.1292052 ],
[-0.14146481, -0.03690118, 0.01252019, -0.03329113, -0.04933522,
-0.11731218, 0.0559076 , -0.02044167, 0.00330163, 0.09573288],
[-0.10201439, -0.099518 , -0.01187718, 0.00162681, -0.02992077,
-0.0786448 , 0.051286 , -0.06118634, 0.00275268, 0.06347791],
[-0.147522 , -0.08012195, -0.06118701, 0.01037565, -0.18682684,
-0.07291215, -0.0621169 , -0.2134946 , 0.0115917 , 0.05180335],
[-0.1349751 , -0.10627404, 0.0418245 , -0.00094373, -0.06426951,
-0.0405674 , 0.0640505 , -0.07173115, 0.03884885, 0.08493583],
[-0.11075551, 0.01280956, 0.04152098, -0.05378193, -0.11458792,
0.05703482, -0.0261404 , -0.08799742, 0.03664519, 0.08952426],
[-0.10952432, 0.03550581, 0.03929392, -0.06884129, -0.08456698,
0.02484545, 0.05707142, -0.03095256, 0.01159266, 0.12775075],
[-0.08891487, -0.00385998, 0.05764434, 0.01283755, 0.02297236,
-0.05666835, 0.07085629, -0.04392565, -0.03329669, 0.07612459],
[-0.08329381, -0.07080178, 0.00815504, 0.00906618, -0.11345821,
-0.05067082, -0.00805459, -0.13532314, 0.00762987, 0.01686591],
[-0.06216974, -0.01758932, -0.03554368, 0.07210717, -0.05302424,
-0.01295669, 0.02067296, -0.02313684, 0.03608295, 0.00777723],
[-0.1243424 , -0.04841175, -0.01277285, -0.06262583, -0.06528353,
-0.03253219, -0.00823232, -0.12434372, -0.06641766, 0.02762409],
[-0.1006383 , -0.03108225, 0.09522864, 0.06852281, -0.12176261,
-0.00717219, -0.11136451, -0.21335904, 0.0541594 , 0.03812924],
[-0.10511756, -0.00343833, 0.04349217, -0.1010148 , -0.1797251 ,
0.06012113, -0.02771915, -0.14802359, 0.00668046, 0.02644532],
[-0.18265271, 0.0074436 , 0.15756166, -0.16698822, -0.09061152,
-0.01037155, 0.07488161, -0.12930974, -0.05729635, 0.0257222 ],
[-0.08828945, 0.03742645, 0.01230084, -0.05811586, -0.08923706,
-0.03776842, 0.05069286, -0.07502478, 0.05130205, -0.00720166],
[-0.19962174, -0.12527527, -0.11753023, 0.08547968, -0.19664416,
-0.04408019, 0.01896041, -0.19835493, 0.0883183 , -0.00914158],
[-0.10420421, -0.09440188, -0.00555003, 0.07209811, -0.21488513,
0.00424226, -0.01504242, -0.13999683, -0.03472971, 0.09850094],
[-0.12265588, -0.00578194, 0.06841818, 0.00064307, -0.10730481,
-0.03361274, 0.02425552, -0.05897643, -0.05878486, 0.05479153],
[-0.02105921, -0.02802122, -0.00410685, 0.05836292, 0.00792775,
-0.06748834, 0.05439955, -0.0800738 , 0.07076354, 0.0353256 ],
[-0.20247069, 0.05606337, 0.09135321, -0.0990632 , -0.02872711,
0.02879303, 0.0678789 , -0.04549425, -0.0880492 , 0.09453399],
[-0.13742954, -0.05233891, 0.07281114, -0.02422397, -0.10600208,
-0.04448827, 0.03444343, -0.19156474, -0.0326518 , 0.04829821],
[-0.06381147, -0.0178546 , -0.04725587, 0.03276172, -0.00874162,
-0.01314383, -0.01579431, -0.06703444, 0.00263928, 0.09629239],
[-0.03175152, -0.07367486, 0.05299102, -0.04229056, -0.12939109,
-0.07581931, 0.00742555, -0.10735898, 0.05252895, 0.02685509],
[-0.1487407 , -0.06161531, 0.03178671, 0.10812385, -0.11724687,
-0.02248264, 0.04004499, -0.13110389, -0.10141636, 0.06114275],
[-0.1841462 , 0.07839277, 0.05206858, -0.08193772, -0.0291815 ,
0.0373794 , 0.07929372, -0.16178085, -0.0498929 , 0.01465369],
[-0.1280095 , -0.10857837, 0.00629766, 0.0297734 , -0.09656814,
-0.02886558, 0.02942573, -0.09436665, -0.00237303, 0.09488904],
[-0.03140139, -0.05389794, 0.0563304 , 0.06379209, 0.04839167,
-0.03151129, 0.0158487 , -0.00435146, -0.03108561, 0.04740969],
[-0.02056921, -0.08638585, 0.09112146, 0.01085307, -0.03224964,
-0.04075257, 0.07115739, -0.09994879, -0.01856763, -0.03763481],
[-0.07598573, -0.05321526, 0.02479915, -0.04398152, -0.0717224 ,
-0.03346183, -0.04348726, -0.05639623, -0.02985962, 0.10864457],
[-0.14914945, -0.01523018, 0.04055313, -0.04227638, -0.07096767,
0.02512853, -0.02308453, -0.16042957, -0.02929157, 0.0873892 ],
[-0.1199998 , -0.00423399, 0.01840039, -0.01986048, -0.16550508,
0.01418665, 0.03537009, -0.0712847 , 0.01129677, 0.05957196],
[-0.22357881, 0.06143619, 0.05910965, -0.05865066, -0.11463878,
0.07018107, 0.00910877, -0.12967779, -0.04628986, 0.03582975],
[-0.05519389, -0.0528973 , 0.00737347, 0.05639487, -0.08122567,
-0.08855919, 0.02301625, -0.10682379, 0.03774085, 0.03102252],
[-0.20162272, 0.06977166, 0.08338684, -0.14451063, -0.12687023,
0.05475996, 0.00907202, -0.18620013, -0.00808838, 0.01913921],
[-0.09057812, 0.0024925 , 0.07000458, -0.07325187, -0.07733104,
-0.00845958, 0.0366439 , -0.02949267, -0.05372918, 0.06086108],
[-0.16978166, -0.17392054, 0.00185057, 0.03467006, -0.30263913,
-0.00441961, -0.11254709, -0.2355696 , 0.04856438, 0.03915354],
[-0.10409875, -0.05012665, 0.02043112, -0.02949139, -0.00657176,
-0.1229246 , 0.04610194, -0.02851345, 0.00120295, 0.07744045],
[-0.13583842, -0.00915503, -0.01576844, -0.04046339, -0.14409027,
-0.01925355, 0.05732079, -0.01254229, 0.02614252, 0.03216061],
[-0.05881466, -0.05291711, 0.04882586, -0.06568014, -0.11591509,
-0.04173259, -0.08914959, -0.18105438, 0.0073473 , 0.07515443],
[-0.09116296, -0.01571628, 0.09542372, -0.04229715, 0.01914682,
0.07099728, 0.05870892, -0.02393101, -0.06685152, 0.14607887],
[-0.01420991, -0.07241911, 0.09915935, -0.0428259 , -0.02433448,
-0.04682102, 0.04740375, -0.0498169 , -0.0381465 , 0.04719302],
[-0.06977817, -0.0549986 , 0.12715016, 0.03751863, -0.05665444,
-0.01156503, 0.02192075, -0.10019331, -0.09799661, 0.07674114],
[-0.12175632, -0.01346524, 0.06833091, 0.06674427, -0.08815148,
-0.01762121, 0.03712306, -0.11569904, -0.04299709, 0.0782926 ],
[-0.1446371 , -0.07632558, -0.08415712, 0.06200639, -0.22020343,
-0.02608814, -0.08402974, -0.25832072, 0.09710971, 0.03165707],
[-0.09859657, -0.06279226, 0.04503319, -0.08128047, -0.02594848,
-0.12064872, -0.03023694, -0.01481692, -0.07295675, 0.07432467],
[-0.14121625, -0.09247054, 0.01716578, -0.02280503, -0.07195783,
-0.07934062, 0.06636599, -0.1391973 , -0.0006269 , 0.10430331],
[-0.1895326 , -0.06477074, -0.0247532 , 0.07115956, -0.12306482,
-0.04250586, 0.04314227, -0.09697549, -0.11742847, 0.11626178],
[-0.17066488, -0.08983296, -0.01518444, 0.14081804, -0.24952582,
0.05580132, -0.04592376, -0.239063 , 0.01135689, 0.05648043],
[-0.0763633 , -0.0462661 , -0.00959867, -0.04214037, -0.1578227 ,
0.03959932, -0.00310759, -0.1926229 , -0.02226702, -0.01654653],
[-0.16822992, -0.15396877, -0.05668662, 0.03996664, -0.21325612,
-0.05039646, -0.04931509, -0.22258577, -0.06086416, 0.1119535 ],
[-0.14810058, -0.06507328, -0.00107521, -0.08621114, -0.05172976,
-0.11022735, 0.08426714, -0.01940702, 0.04058618, 0.0811009 ],
[-0.14031763, 0.01446593, 0.07438764, -0.01220013, -0.10643414,
0.03723364, 0.07789111, 0.0064223 , -0.02987565, 0.05971198]],
dtype=float32)>,
1: <tf.Tensor: shape=(64, 10), dtype=float32, numpy=
array([[-2.26268619e-01, -2.28828471e-02, 8.45070630e-02,
-1.76083110e-02, -1.29523739e-01, 1.58308834e-01,
6.23306856e-02, -1.73144519e-01, -3.93079296e-02,
1.17382959e-01],
[-1.97477624e-01, -7.75161907e-02, -8.48741233e-02,
1.13340229e-01, -2.41238594e-01, 1.21705700e-02,
-7.02917874e-02, -2.78167903e-01, 1.19012177e-01,
8.45900476e-02],
[-1.66494250e-01, 1.02631869e-02, -5.66720404e-02,
-5.38627282e-02, -1.36432394e-01, -5.16337156e-03,
5.12326919e-02, -1.59885526e-01, -1.34082288e-02,
1.21314190e-01],
[-4.16232198e-02, -3.15627083e-04, -2.59173959e-02,
-1.20904958e-02, -6.11409657e-02, 4.05119807e-02,
4.19395529e-02, -2.51966976e-02, 7.27809221e-02,
1.80586968e-02],
[-2.39562020e-01, -1.98421702e-01, -4.99575958e-02,
7.92631954e-02, -2.58657932e-01, -1.17474291e-02,
-5.35634346e-02, -2.41889566e-01, 2.02101134e-02,
1.42651290e-01],
[-1.47032171e-01, 2.86730491e-02, -2.96947844e-02,
4.12945747e-02, -1.29176736e-01, -5.81263527e-02,
6.80094212e-03, -1.12203687e-01, 2.99334154e-02,
1.58465151e-02],
[-9.87898111e-02, 1.40235443e-02, 9.44875367e-03,
-1.67544633e-02, -1.09810486e-01, 6.90223724e-02,
5.52554391e-02, -1.51135817e-01, -1.29527738e-02,
-1.05757397e-02],
[-1.59293711e-01, -1.16223425e-01, -2.69010663e-02,
-3.74424011e-02, -1.04160830e-01, -1.18797362e-01,
-4.02004272e-03, -3.31426561e-02, -5.19968942e-02,
1.71776801e-01],
[-4.13588956e-02, -6.86522108e-03, -1.23194773e-02,
-8.37539583e-02, -9.79086086e-02, -6.04871288e-02,
4.46226858e-02, -9.47351232e-02, 5.69011793e-02,
3.06082349e-02],
[-1.09332033e-01, 6.70426339e-03, 7.12826326e-02,
1.88035294e-02, -6.99934959e-02, -3.66151631e-02,
8.98275599e-02, -9.86446887e-02, -5.97077422e-03,
7.63730556e-02],
[-9.65768024e-02, 2.13371180e-02, 3.16816643e-02,
9.87571292e-03, -9.23058838e-02, 2.85959914e-02,
5.84699064e-02, 1.24091469e-02, 4.08947393e-02,
4.90225554e-02],
[-1.82662547e-01, 7.01846927e-03, 6.68549724e-03,
-3.09559703e-03, -1.40156671e-01, -4.12398949e-02,
1.33544579e-03, -1.15964614e-01, -1.11097842e-03,
-1.39631163e-02],
[-1.90077692e-01, 3.64055634e-02, 5.36940843e-02,
8.03370774e-03, -6.85881525e-02, 8.35565254e-02,
1.07485019e-01, -3.04609984e-02, 2.47915741e-03,
1.07842855e-01],
[-1.24872848e-01, -4.90949787e-02, 2.85280272e-02,
-9.26144421e-02, 1.75680891e-02, -1.23154938e-01,
1.82249583e-02, -7.06329718e-02, -1.79566965e-02,
1.72657222e-02],
[-4.24651839e-02, 8.96332972e-03, 7.87791014e-02,
2.58662850e-02, -1.07430644e-01, 7.82843307e-02,
4.42989692e-02, -1.11840934e-01, 4.40142974e-02,
-2.38031112e-02],
[-7.26791173e-02, -4.98233885e-02, 1.04386240e-01,
-3.65755446e-02, -5.39079159e-02, -2.43936982e-02,
8.03450644e-02, -1.17534967e-02, 1.06069110e-02,
4.99468297e-02],
[-1.12117320e-01, -8.98748934e-02, 3.97989713e-02,
9.28172693e-02, -1.77596256e-01, -7.08229616e-02,
-2.51212604e-02, -1.95586935e-01, 8.37485343e-02,
-1.40025988e-02],
[-1.58346921e-01, -8.24693590e-02, -5.65039590e-02,
6.45706281e-02, -1.74745858e-01, -3.12444977e-02,
-6.05197065e-02, -2.20325321e-01, 2.59073079e-02,
3.20266448e-02],
[-1.58618331e-01, -3.79656516e-02, 6.72528446e-02,
6.88800365e-02, -9.66797322e-02, -7.46569410e-03,
9.68661606e-02, -1.48422182e-01, -2.02783216e-02,
3.89614664e-02],
[-8.94666016e-02, -7.58211538e-02, 6.62114546e-02,
2.82529220e-02, -6.23007305e-02, -1.28947303e-01,
6.08180761e-02, -1.87309496e-02, 3.45235094e-02,
4.35458533e-02],
[-9.85482559e-02, 3.30734625e-02, 5.59863113e-02,
-2.70094108e-02, -6.41853139e-02, 7.15028122e-02,
9.02910344e-03, -9.60181206e-02, -7.73606002e-02,
8.58743563e-02],
[-1.97032653e-02, -2.92284843e-02, 1.02460273e-02,
-1.02846496e-01, -9.06485394e-02, -2.00703926e-02,
1.99896693e-02, -5.88451996e-02, 4.92874160e-02,
7.08697736e-03],
[-1.50538355e-01, -8.45866054e-02, 5.64673506e-02,
1.86246522e-02, -8.48133788e-02, -8.47089067e-02,
7.24234134e-02, -6.05773814e-02, -6.78267181e-02,
5.83692826e-02],
[-1.22482985e-01, 8.11095461e-02, 6.29960150e-02,
-6.08337037e-02, -9.54639614e-02, 3.43603566e-02,
3.36722136e-02, -1.48602888e-01, 1.49103440e-03,
3.74073237e-02],
[-1.09655358e-01, -7.25264400e-02, -1.18080322e-02,
-5.90124018e-02, -1.32899567e-01, -1.52138527e-02,
-5.00685126e-02, -8.11381191e-02, 2.29290836e-02,
7.48161003e-02],
[-6.26883209e-02, -6.46792054e-02, 1.08280797e-02,
-1.76118016e-02, -1.00234866e-01, -9.46881026e-02,
-3.74057889e-02, -1.64193183e-01, 3.44156884e-02,
1.39950141e-02],
[-5.57693467e-02, -1.30172959e-02, 5.79589121e-02,
1.86375007e-02, -4.27588299e-02, -7.55617768e-02,
6.05458207e-02, -1.03945538e-01, -7.66253099e-04,
5.73797971e-02],
[-1.10689014e-01, -1.14388064e-01, 2.85286717e-02,
8.50981399e-02, -3.72097529e-02, 5.57603501e-03,
-2.23728791e-02, -9.16901082e-02, -4.54919264e-02,
4.19731401e-02],
[-5.13254851e-02, 2.36552879e-02, 1.97334606e-02,
-2.73641124e-02, -7.77815953e-02, 5.02437353e-02,
4.55379114e-02, -2.94566955e-02, 2.43971646e-02,
5.05007505e-02],
[-5.17050214e-02, 6.95329346e-03, -6.85334485e-03,
-2.24254578e-02, -6.58822656e-02, 1.99540164e-02,
5.30460477e-02, -2.02816520e-02, 3.22487727e-02,
3.30003351e-02],
[-1.12133816e-01, -2.69253477e-02, -1.59975477e-02,
2.42674537e-02, -1.00763790e-01, 4.60313037e-02,
7.73231387e-02, -1.45861298e-01, 2.86127720e-03,
5.42693883e-02],
[-1.62743062e-01, -7.25711090e-03, 8.25979784e-02,
-2.98158824e-03, -7.24797025e-02, -9.59400088e-03,
1.03634164e-01, -5.32378033e-02, -7.22575262e-02,
6.62382469e-02],
[-1.74668640e-01, -1.33392755e-02, -1.62338726e-02,
-4.47108820e-02, -4.97036204e-02, -1.84731521e-02,
4.78231385e-02, -5.82275987e-02, 9.64488648e-03,
9.01553184e-02],
[-1.03768170e-01, 5.39408997e-02, 9.83848721e-02,
3.93026024e-02, -1.02525547e-01, 1.68007463e-02,
4.22611870e-02, -1.77876413e-01, -8.41764882e-02,
1.51371248e-02],
[-8.46128538e-02, 9.90282558e-03, 3.06268018e-02,
1.05842054e-02, -3.39219011e-02, 7.39571899e-02,
3.69111039e-02, -6.83671609e-03, 2.81678494e-02,
1.04405962e-01],
[-9.96439382e-02, -5.52335344e-02, -3.63139436e-02,
-2.96811014e-02, -1.63912863e-01, -1.37102097e-01,
-6.49546683e-02, -1.43455938e-01, -6.13795966e-03,
-1.51403025e-02],
[-1.15472242e-01, 5.99557124e-02, -3.01486887e-02,
-5.95454164e-02, -8.94712061e-02, -2.32638139e-02,
6.58176318e-02, -6.57178462e-02, 3.96647081e-02,
1.36262011e-02],
[-2.09447712e-01, -9.19480547e-02, 2.48690564e-02,
9.96455252e-02, -2.31332466e-01, 1.26705505e-03,
-6.75062388e-02, -2.28245527e-01, -2.45758891e-02,
4.23956439e-02],
[-8.25200677e-02, -4.79681045e-02, 6.78283721e-02,
6.95098937e-03, -1.85745552e-01, 5.93830273e-02,
3.72107476e-02, -1.25796080e-01, -8.67868066e-02,
5.97448647e-02],
[-1.35011613e-01, -1.04279891e-01, -1.93549227e-02,
4.67273071e-02, -5.95514327e-02, -1.26426160e-01,
4.73011211e-02, -3.98379453e-02, -3.08366567e-02,
1.55897036e-01],
[-1.90755218e-01, -1.24813542e-02, 8.11097771e-02,
-4.41103168e-02, -2.93283425e-02, -5.21741062e-02,
3.55289541e-02, -4.78535667e-02, -9.69170183e-02,
8.22419375e-02],
[-1.32893249e-01, 1.86000252e-03, 7.29389340e-02,
2.08595060e-02, -4.58938703e-02, -6.47663027e-02,
7.13512301e-02, -5.47017232e-02, 1.56622566e-03,
-2.21497715e-02],
[-1.68044597e-01, 1.67937763e-02, 3.23309563e-02,
3.25734168e-03, -5.41884080e-02, -2.38925405e-03,
6.79029301e-02, -8.15764815e-02, -8.97403061e-02,
1.59459889e-01],
[-1.75073147e-02, 1.23760737e-02, 9.89615321e-02,
-2.04793699e-02, 4.66362461e-02, 3.67780738e-02,
5.67866787e-02, -3.37878503e-02, 4.00665998e-02,
9.95308459e-02],
[-5.65849096e-02, -1.39236711e-02, 3.58471647e-03,
4.47895192e-03, -2.11294200e-02, 6.35869056e-02,
3.07296775e-02, 1.08992560e-02, 4.49587405e-02,
4.75549549e-02],
[-1.39448419e-01, -4.56879474e-02, -2.65447609e-02,
-2.23840624e-02, -5.82361743e-02, -1.45420402e-01,
3.09186969e-02, -1.04892500e-01, -2.75357980e-02,
1.06248930e-01],
[-8.21909755e-02, 1.93269439e-02, 5.26269786e-02,
-2.53011882e-02, -2.15167981e-02, -6.39346913e-02,
1.01416782e-01, -8.70964229e-02, 2.92393602e-02,
7.91490003e-02],
[-1.74655527e-01, -1.21824611e-02, 1.01610713e-01,
-5.85027784e-02, -6.69525117e-02, 2.52785366e-02,
-1.28451493e-02, -1.68689817e-01, -9.84297916e-02,
-1.53103955e-02],
[-1.48807019e-01, -4.84034382e-02, -1.88913923e-02,
-1.22408271e-02, -7.77565241e-02, -1.54015012e-02,
3.45174819e-02, -9.77065880e-03, 3.78726274e-02,
9.88224968e-02],
[-6.38555214e-02, -1.28440727e-02, 4.78540137e-02,
-1.69302318e-02, -1.01742893e-01, 3.53637449e-02,
4.99530286e-02, 1.15835033e-02, 8.70382488e-02,
2.12726910e-02],
[-1.17232144e-01, -1.25898391e-01, 1.30450409e-02,
2.20539719e-02, -1.93232402e-01, -4.28303033e-02,
-3.89161706e-02, -2.24561840e-01, -1.81237496e-02,
1.95829496e-02],
[-9.51445699e-02, -3.76937725e-02, 1.33399621e-01,
7.20417127e-03, -4.53586467e-02, 6.13040812e-02,
-1.83444060e-02, -5.56321107e-02, -3.60671207e-02,
5.61316013e-02],
[-8.68022293e-02, 1.20465020e-02, -6.54198602e-03,
-2.95846239e-02, -1.05969362e-01, 4.36754040e-02,
6.49331883e-02, -1.46021754e-01, -1.90029591e-02,
4.40182202e-02],
[-1.00222915e-01, -1.25300102e-02, 2.65474599e-02,
-2.45862342e-02, -2.26879045e-02, 3.97172198e-03,
8.50273743e-02, -7.38859549e-02, -5.95477261e-02,
9.78583843e-02],
[-7.15610608e-02, -9.33187604e-02, 4.55188900e-02,
-2.19681785e-02, -8.24558064e-02, -1.12349153e-01,
8.78680497e-03, -1.79894015e-01, -3.21138687e-02,
1.68573782e-02],
[-1.06039301e-01, -4.53191660e-02, 4.70333099e-02,
5.05055264e-02, -1.09207615e-01, -3.28311212e-02,
-1.72384083e-03, -1.68462887e-01, -7.24901706e-02,
6.98208511e-02],
[-1.77585199e-01, 1.36083737e-03, 8.73919651e-02,
-1.12462267e-02, -1.28724426e-01, 6.29377179e-03,
7.31705800e-02, -1.07890323e-01, -6.39854372e-02,
4.20372188e-02],
[-1.87925667e-01, -5.57832904e-02, -9.57410336e-02,
-2.83403918e-02, -2.61434972e-01, -3.26225534e-02,
-1.92749277e-02, -2.70305395e-01, 4.04147953e-02,
1.26160383e-02],
[-9.38892886e-02, -1.15765870e-01, 9.65594351e-02,
-3.07104513e-02, -4.42704596e-02, -9.61312205e-02,
3.35111320e-02, -9.12012532e-02, 4.88444790e-02,
9.73151475e-02],
[-9.06076357e-02, -4.03449684e-02, 5.05531915e-02,
3.13387960e-02, -1.06617197e-01, 1.05631359e-01,
7.63250142e-02, -1.63521126e-01, -6.81229606e-02,
4.49142680e-02],
[-1.00606956e-01, 2.33182702e-02, 1.09717906e-01,
-3.46118547e-02, 2.17448846e-02, 1.89216007e-02,
1.61103547e-01, -4.53322753e-02, 4.63520959e-02,
1.22339606e-01],
[-3.73194851e-02, -8.53166636e-03, 3.89622673e-02,
-2.62460243e-02, -1.47193521e-02, -9.89666395e-03,
6.01450913e-03, 2.45102197e-02, 1.95712969e-02,
-2.51837075e-04],
[-1.08675152e-01, 3.31245065e-02, 7.45619982e-02,
-1.60338394e-02, -1.22897953e-01, 3.78145315e-02,
7.88810551e-02, -7.27499574e-02, 2.92414650e-02,
-8.81527364e-03],
[-1.52145386e-01, 8.87341201e-02, 6.57501817e-02,
-1.19503289e-01, -1.41874179e-02, 5.17417677e-03,
1.47610903e-03, -7.63006434e-02, -4.17422876e-02,
1.02013886e-01]], dtype=float32)>,
2: <tf.Tensor: shape=(64, 10), dtype=float32, numpy=
array([[-1.59998268e-01, 1.00655761e-02, 3.19059268e-02,
-8.82465988e-02, -1.47808343e-01, -3.22506838e-02,
1.67885236e-02, -1.78845942e-01, -9.62061062e-03,
-1.13258809e-02],
[-1.23316877e-01, -5.22477776e-02, 8.76506045e-02,
1.85575336e-04, -2.46180538e-02, -2.07363367e-02,
-5.80711141e-02, -9.60124880e-02, -1.00168921e-01,
3.40207368e-02],
[-1.59052879e-01, -7.41913468e-02, 3.14843431e-02,
7.71991163e-02, -6.57074600e-02, -1.01665768e-03,
4.79415432e-02, -1.76090553e-01, 1.12063903e-03,
1.43461362e-01],
[-4.49170694e-02, 1.54931694e-02, 4.42449078e-02,
-7.39493519e-02, -6.89156577e-02, -4.57458161e-02,
1.53775141e-02, -1.37762994e-01, -5.71796298e-03,
3.84313948e-02],
[-5.66461906e-02, 4.07216139e-03, -5.39162010e-02,
-5.73162846e-02, -6.25002310e-02, -3.69265638e-02,
4.21732888e-02, -4.31450456e-02, -6.56469259e-03,
6.87618628e-02],
[-1.57791555e-01, -2.21033841e-02, 9.01331753e-02,
6.99925795e-02, -1.13660648e-01, -3.31985578e-02,
-6.04883395e-03, -2.21236661e-01, -5.13135530e-02,
-3.74600738e-02],
[-1.17652573e-01, 6.01082593e-02, 6.99851438e-02,
-8.96160230e-02, -6.91386499e-03, -3.06419469e-03,
2.96859071e-02, -7.44917616e-03, -6.35416955e-02,
9.03566927e-02],
[-1.09585039e-01, -8.90047252e-02, 6.56725839e-04,
-2.27427967e-02, -6.28467649e-02, 1.72538236e-02,
5.54561615e-03, -7.12843686e-02, -2.11970843e-02,
1.46655127e-01],
[-6.79751262e-02, -1.32717062e-02, 5.77747300e-02,
-4.06415015e-03, -1.00256696e-01, 2.82425173e-02,
8.65132511e-02, 1.35657918e-02, 8.03826600e-02,
2.75349170e-02],
[-3.32130045e-02, 4.31914628e-02, -7.93108344e-03,
-7.78385252e-02, -8.39967057e-02, 5.77804223e-02,
4.68557328e-02, -8.02627727e-02, -2.17813421e-02,
5.66784814e-02],
[-1.37984574e-01, -2.40551680e-03, 6.58576488e-02,
-6.24554306e-02, -7.69103318e-02, 5.28060459e-02,
5.43180108e-03, -7.75522292e-02, -2.59037670e-02,
9.57907587e-02],
[-1.06047109e-01, 3.06454897e-02, 1.02165438e-01,
-8.11553001e-03, -6.25192374e-02, -9.56596434e-03,
7.58369267e-02, -1.39230281e-01, -2.87407935e-02,
1.99384764e-02],
[-8.18956569e-02, -4.10744995e-02, 2.59871054e-02,
-6.70721531e-02, -8.04064199e-02, -4.73484024e-02,
5.31727448e-02, -1.18960671e-01, 2.24236213e-02,
2.59504505e-02],
[-1.63176402e-01, 1.69006828e-02, -1.29912533e-02,
-7.71650523e-02, -1.15306251e-01, 2.68699639e-02,
1.84115060e-02, -6.85939565e-02, 2.56212614e-02,
6.76540583e-02],
[-1.44753516e-01, 9.24872905e-02, 1.30240262e-01,
-8.11262280e-02, 5.47917932e-03, -1.81658939e-03,
1.00948200e-01, -1.55141875e-01, -8.17784369e-02,
-3.31878439e-02],
[-2.86125213e-01, -9.03276503e-02, 4.85181771e-02,
1.20044097e-01, -3.07462573e-01, 7.69088492e-02,
-3.61254700e-02, -2.52649307e-01, -3.91766019e-02,
4.97994833e-02],
[-7.90339410e-02, 5.87714463e-02, 2.68070549e-02,
-5.54916076e-02, -9.72821712e-02, 1.31056309e-02,
4.45548333e-02, -1.25619918e-01, 7.66982511e-03,
7.28771314e-02],
[-1.42986119e-01, 6.70846850e-02, -4.65226546e-03,
-5.62149733e-02, -1.17274195e-01, -6.90244138e-04,
1.19525559e-01, -5.04034534e-02, 4.85993475e-02,
4.73382398e-02],
[-1.72590390e-01, -5.34471050e-02, -1.97447315e-02,
2.48760693e-02, -9.63888839e-02, -2.33638976e-02,
8.43596458e-02, -2.83525698e-03, 4.59408760e-03,
7.87654221e-02],
[-1.63182750e-01, 6.14318885e-02, 5.54641746e-02,
-5.89274615e-03, -1.33169144e-01, 1.17879689e-01,
6.14799485e-02, -1.59166008e-01, -6.00537322e-02,
1.08472966e-02],
[-1.75962359e-01, -1.19502850e-01, 5.22347242e-02,
7.63762966e-02, -1.57802194e-01, -6.54878169e-02,
-9.30812284e-02, -1.74125910e-01, -3.19207832e-03,
-3.91837358e-02],
[-1.68669865e-01, 1.72981396e-02, -9.75131895e-03,
1.09532923e-02, -1.12948187e-01, -7.92109072e-02,
2.93207970e-02, -9.22475979e-02, 1.88220590e-02,
-9.40106437e-03],
[-1.37932956e-01, 1.85102001e-02, -1.91209912e-02,
-4.04065624e-02, -1.44183666e-01, 4.93810419e-03,
7.42893592e-02, -4.71776463e-02, 6.51464015e-02,
4.86958176e-02],
[-1.74774021e-01, 5.98106906e-02, 1.19641855e-01,
3.01058590e-03, -4.78949547e-02, 1.10694885e-01,
6.57292753e-02, -7.00315088e-02, -1.75423324e-02,
1.91642895e-01],
[-2.06818312e-01, -4.49922718e-02, 9.38108042e-02,
-1.82551928e-02, -5.91449291e-02, 5.88195771e-02,
1.72943883e-02, -1.75111115e-01, -2.13059708e-02,
5.26921973e-02],
[-1.74163356e-01, 3.60969603e-02, 1.46923751e-01,
-1.22066483e-01, -1.46349128e-02, -3.29729877e-02,
9.39526409e-02, -1.32425249e-01, -1.20000333e-01,
1.19355991e-02],
[-1.34692788e-01, 4.38029431e-02, 6.75951988e-02,
-4.23112772e-02, -9.91474465e-02, 1.46606583e-02,
8.13097879e-02, -7.40392357e-02, 9.16860998e-04,
8.00481439e-03],
[-1.14981219e-01, -3.37088434e-03, 1.07005276e-02,
3.33579406e-02, -5.95754012e-02, -5.68026267e-02,
3.35185565e-02, -1.21869668e-01, 3.44074890e-02,
8.71154666e-03],
[-1.36970699e-01, -1.00058906e-01, -4.92304489e-02,
5.70428595e-02, -1.72202483e-01, 2.26514153e-02,
-4.75450978e-02, -2.28004366e-01, 8.70785341e-02,
1.53522007e-02],
[-1.55105978e-01, -1.50688943e-02, 3.62236425e-03,
4.44897003e-02, -1.38242334e-01, -4.65613641e-02,
3.33304293e-02, -9.56235528e-02, -3.43733653e-02,
4.26802486e-02],
[-7.41859600e-02, -2.15787962e-02, -2.58451328e-04,
-5.96803874e-02, -1.03493266e-01, -1.26781225e-01,
-3.56705766e-03, -1.73495740e-01, 1.24403555e-02,
5.54167479e-03],
[-4.76967283e-02, -5.29718846e-02, -1.77503955e-02,
-1.52167901e-02, -1.06719553e-01, -1.59110613e-02,
4.33917269e-02, -8.67965519e-02, 4.73607667e-02,
6.17296621e-02],
[-1.82744563e-01, 3.83751802e-02, 2.01678462e-03,
-8.37065428e-02, -1.27897531e-01, 3.02882921e-02,
5.21660596e-02, -1.18030235e-01, -2.59065777e-02,
6.20141216e-02],
[-6.95300326e-02, -1.75547600e-02, -3.17562036e-02,
4.39015031e-02, -1.14880711e-01, -8.48406181e-02,
1.90095082e-02, -1.37437284e-01, 9.43323225e-03,
9.49483216e-02],
[-8.87794718e-02, 1.36397779e-03, -1.81018133e-02,
3.90491709e-02, -1.42404109e-01, 3.30205448e-03,
-2.97277309e-02, -1.70011967e-01, 9.32460576e-02,
4.40285653e-02],
[-2.11009666e-01, -5.76778352e-02, -3.60394195e-02,
-1.06923096e-02, -3.97471711e-02, -1.14382111e-01,
4.56975698e-02, -1.08936004e-01, 7.08763488e-03,
3.75104770e-02],
[-1.29108429e-01, -1.37775168e-02, 1.11769643e-02,
1.55910924e-02, -7.88502470e-02, -4.01405096e-02,
5.04296236e-02, -1.18844166e-01, -5.63968495e-02,
4.68248613e-02],
[-5.63661903e-02, -2.07510777e-02, 7.93255940e-02,
-5.05831838e-03, -8.02241117e-02, 5.22273332e-02,
-2.28450540e-02, -1.06059760e-01, -1.02242626e-01,
1.49079621e-01],
[-1.19903348e-01, -2.67150104e-02, 1.24713048e-01,
-1.15524232e-03, -1.23427682e-01, -4.63337637e-02,
1.92754157e-02, -1.66648015e-01, -3.70079949e-02,
7.55362213e-02],
[-1.11221313e-01, 4.92718816e-02, -2.06203461e-02,
-6.61591366e-02, -8.08398873e-02, 7.35159591e-03,
3.56219485e-02, -1.41128287e-01, -2.54322439e-02,
3.59114222e-02],
[-4.30217497e-02, -5.75315580e-02, 1.57062896e-03,
-3.38286720e-02, -6.17156923e-02, -6.39270246e-03,
6.76650181e-02, -9.69059616e-02, 4.56762500e-03,
7.08248839e-02],
[-4.02978510e-02, -1.06733106e-01, -5.11328364e-03,
6.64058775e-02, -9.06679034e-02, -6.77974746e-02,
2.62845661e-02, -1.61776602e-01, 9.83053148e-02,
-1.44332536e-02],
[-1.99293256e-01, -8.44378583e-03, 3.82554531e-02,
-7.23803118e-02, -1.47957891e-01, 9.17248502e-02,
-2.19110511e-02, -2.20367521e-01, -1.07786059e-03,
9.08675790e-02],
[-6.16713054e-02, 2.18773969e-02, 1.46675020e-01,
-6.06631674e-02, -4.59251590e-02, 2.01707222e-02,
1.52159780e-02, -9.58537683e-02, -4.12879512e-02,
1.12033114e-01],
[-2.21959561e-01, 3.75759527e-02, 1.25662223e-01,
-7.22270459e-02, -5.28829768e-02, 2.52887234e-03,
4.14352380e-02, -1.38201237e-01, -8.72305483e-02,
1.62369609e-02],
[-6.13061711e-02, -1.10067680e-01, 6.17929846e-02,
2.61496585e-02, -2.95830965e-02, -8.19942206e-02,
1.36333480e-01, -7.25157782e-02, 3.05302218e-02,
-2.78179571e-02],
[-8.49346966e-02, -9.67465192e-02, 3.63172144e-02,
-2.42773518e-02, -1.83681607e-01, 2.14811713e-02,
-4.00538370e-02, -1.64074063e-01, -7.49438852e-02,
1.29477248e-01],
[-1.93623543e-01, -1.31538093e-01, 3.41942981e-02,
1.32095039e-01, -1.28099456e-01, 3.54318619e-02,
-8.16837698e-02, -1.99805722e-01, -7.68171996e-03,
6.61233887e-02],
[-1.60500348e-01, -7.89126195e-03, -1.54516473e-03,
4.06349264e-03, -1.31307676e-01, -1.61880292e-02,
7.96549469e-02, 2.03972422e-02, 2.50642337e-02,
5.49629405e-02],
[-4.39041480e-02, 9.65190306e-03, 7.49287568e-03,
-2.05658749e-02, -6.17776439e-02, 1.79488976e-02,
3.93416472e-02, 1.50518157e-02, 5.96523359e-02,
7.97144622e-02],
[-8.73474777e-02, -2.67238691e-02, 4.36430313e-02,
1.96785778e-02, -6.03708364e-02, 3.22387666e-02,
-4.45651859e-02, -1.44783229e-01, -3.73603925e-02,
9.25010443e-02],
[-1.32528991e-01, -1.00225851e-01, 3.47075537e-02,
2.49354355e-02, -8.73976052e-02, -2.39339396e-02,
2.70368010e-02, -9.90849920e-03, -3.44063491e-02,
8.05625468e-02],
[-1.65621281e-01, -5.85396215e-03, 2.49869078e-02,
-1.18815992e-02, -7.31760785e-02, -5.53333424e-02,
5.17196171e-02, 7.82136023e-02, -2.86602974e-03,
4.38763052e-02],
[-1.78106934e-01, 5.95950522e-02, 1.26824975e-01,
-7.42239505e-02, -3.21754143e-02, 2.49740276e-02,
6.33400157e-02, -2.10573554e-01, -7.08340257e-02,
2.46340409e-02],
[-1.06025636e-01, 3.46532185e-03, -2.25651339e-02,
-4.98146415e-02, -7.49360174e-02, -2.30764039e-02,
5.50748110e-02, -4.30858284e-02, 1.24792010e-03,
2.55629532e-02],
[-1.65806234e-01, 1.00251446e-02, 1.51646379e-02,
-4.80565205e-02, -1.57962769e-01, 9.11003817e-03,
4.12575118e-02, -1.33541040e-03, 2.78127808e-02,
4.69618253e-02],
[-1.74214728e-02, -7.85977617e-02, 8.38243216e-02,
-2.98012048e-02, -6.33828789e-02, 6.52255304e-03,
7.31061399e-02, -9.39695984e-02, 6.02594018e-03,
5.51375560e-02],
[-1.18666902e-01, -3.74546982e-02, 2.97008082e-02,
-8.28697905e-03, -6.89347759e-02, 3.38471495e-04,
4.33129929e-02, -9.98683274e-02, -7.06672389e-03,
8.01912919e-02],
[-8.25223029e-02, 3.90745997e-02, 1.06189221e-01,
-2.55585313e-02, -3.45373638e-02, 1.58698589e-01,
1.03521571e-01, 4.37497534e-03, -2.03898698e-02,
1.21164784e-01],
[-7.92142227e-02, -1.41440332e-03, 4.25021537e-02,
-5.23098111e-02, 2.72297114e-03, -8.03528652e-02,
1.47505999e-02, -1.18839061e-02, 1.58419050e-02,
8.25753361e-02],
[-9.94176418e-02, 4.66350988e-02, 3.88142020e-02,
-5.72577566e-02, -3.54181081e-02, -1.63416732e-02,
1.39044067e-02, -8.42689872e-02, -1.73356067e-02,
7.15929195e-02],
[-1.17300324e-01, -2.53385678e-03, 1.22368969e-02,
-6.57753795e-02, -8.61979872e-02, 1.71724372e-02,
2.96129175e-02, -5.80403581e-02, -4.59693279e-03,
1.00424349e-01],
[-7.17606992e-02, 3.73484455e-02, 7.98465610e-02,
-1.13344841e-01, 2.85623968e-02, -7.84002021e-02,
1.14720270e-01, -1.37059335e-02, -4.67590429e-02,
-3.58766019e-02],
[-1.19252682e-01, -5.46724014e-02, 2.12554671e-02,
-2.60256901e-02, -1.18838035e-01, -7.79783353e-04,
1.17120825e-01, -8.98669660e-02, -1.56934150e-02,
7.52980337e-02]], dtype=float32)>,
3: <tf.Tensor: shape=(64, 10), dtype=float32, numpy=
array([[-0.13816997, 0.06492373, 0.05963548, -0.07749565, -0.04870264,
0.03043101, 0.10542686, 0.03885417, 0.0337164 , 0.05817464],
[-0.11279859, -0.04337481, 0.03435868, -0.02655246, -0.11952962,
-0.01797065, 0.08983426, -0.09365702, 0.00677602, 0.04268266],
[-0.10954604, 0.03435408, 0.11179575, -0.03154019, 0.02591707,
0.09995584, 0.08463538, -0.05970678, -0.0848078 , 0.15756509],
[-0.12352699, -0.01917969, 0.09936933, 0.01415488, -0.09542558,
0.00089294, 0.09209705, -0.02983551, -0.02195595, 0.04671702],
[-0.06756619, -0.00489895, 0.0117432 , -0.02592725, -0.04968105,
-0.06794515, 0.0604201 , -0.07477228, 0.00324716, 0.01720311],
[-0.0845879 , 0.04729264, 0.0078808 , -0.0386478 , -0.07596058,
0.05729974, 0.01837927, 0.00918104, 0.05124718, 0.07136449],
[-0.07948235, -0.00960585, 0.04758345, -0.01597166, -0.07711881,
0.02708039, 0.06271845, 0.03696656, 0.08396289, 0.026587 ],
[-0.12800741, 0.01829692, 0.1356301 , -0.06682108, -0.06846179,
0.10691822, -0.02013552, -0.14088607, -0.11650906, 0.13466966],
[-0.08416323, -0.01797319, 0.02912706, -0.05476924, -0.20230307,
0.0882366 , -0.05526718, -0.19650573, 0.09735392, -0.01393362],
[-0.11843733, -0.08165433, 0.02410641, 0.03282395, -0.11065666,
0.0179927 , -0.00462328, -0.14068761, 0.03451999, 0.0669435 ],
[-0.13145748, -0.15158895, 0.01796156, 0.0877454 , -0.13613257,
-0.01723069, -0.07388778, -0.17046481, -0.00131415, 0.08616079],
[-0.08663268, -0.13788612, 0.04096151, 0.11944659, 0.00735119,
-0.08342101, 0.09707905, -0.03895798, 0.07617573, 0.06746137],
[-0.12454629, 0.05573795, 0.0075143 , -0.04392793, -0.13838992,
-0.05890861, 0.04481973, -0.10845333, 0.01106796, -0.00301209],
[-0.16080135, -0.04355218, 0.07966243, 0.01295466, -0.09679088,
0.00747897, 0.03240683, -0.12266912, -0.03876461, 0.07168967],
[-0.13938674, 0.02557349, 0.06166683, -0.03306177, -0.0720713 ,
-0.00631822, 0.00324254, -0.01210807, -0.08504589, 0.10959022],
[-0.0936108 , -0.06967961, 0.06891145, -0.00365001, -0.0345473 ,
-0.01639447, 0.07378035, -0.07014777, -0.00537141, 0.12463517],
[-0.10745381, 0.05816724, 0.12436502, -0.03434642, -0.06851086,
0.10255954, 0.03396888, -0.11661474, -0.08296818, 0.15343796],
[-0.07215579, -0.05064156, 0.10252836, 0.00424037, 0.0493798 ,
-0.03018603, 0.08868306, 0.03668336, 0.00039043, 0.11681915],
[-0.18709819, 0.01972594, 0.09062947, -0.06644759, -0.14828035,
0.06537472, 0.04527552, -0.17170322, -0.06965317, 0.00948366],
[-0.15296176, -0.16802716, 0.00662165, 0.03837484, -0.21782206,
-0.06441768, -0.12475041, -0.21509448, 0.04689124, 0.03159034],
[-0.2532048 , 0.04150939, 0.02173881, -0.04858812, -0.15540019,
0.01055205, 0.07055274, -0.02494052, 0.00246481, 0.06276888],
[-0.11605556, -0.06928153, 0.03512081, -0.08543567, -0.1121366 ,
-0.10300419, 0.07662523, -0.15475114, 0.04059283, 0.02116433],
[-0.104318 , -0.00831181, 0.01996926, -0.03608937, -0.10455424,
0.01077326, 0.06429273, -0.15449709, 0.02032354, 0.00147541],
[-0.02962596, -0.05511342, 0.05481257, -0.03031509, -0.01214547,
0.00701355, 0.03530937, -0.07908271, 0.0375403 , 0.09198669],
[-0.09283669, -0.04342161, 0.02300952, 0.08132045, -0.07933894,
-0.09115605, -0.00496355, -0.10422132, -0.03438527, 0.06268802],
[-0.21061012, -0.01681675, -0.03493425, -0.01469951, -0.12970988,
-0.02512392, 0.07786143, -0.02652392, -0.01397093, 0.06971685],
[-0.19916871, -0.04717059, 0.04001759, 0.02057964, -0.23623674,
0.06024899, 0.06213139, -0.21670324, -0.02990749, 0.0253678 ],
[-0.20721811, 0.02971236, 0.07693379, -0.04177717, -0.02486595,
0.03127091, 0.06497063, -0.14871366, -0.12440205, 0.07020485],
[-0.08742791, -0.00075644, 0.10997505, -0.01346361, -0.09975891,
0.06077233, 0.01937567, -0.12241828, 0.0441133 , 0.03294979],
[-0.23620988, -0.08286667, 0.02880071, 0.03093151, -0.03544433,
-0.05640813, 0.02596629, -0.03254943, -0.12279023, 0.02745904],
[-0.12076453, -0.07451122, 0.02921418, 0.00548238, -0.02276401,
-0.05666757, -0.02516788, -0.03047131, -0.02932606, 0.04760972],
[-0.23152424, -0.06616215, -0.03868683, 0.10819355, -0.28082877,
0.11079868, -0.0655142 , -0.23609804, -0.04490621, 0.0523609 ],
[-0.13345924, -0.07582654, -0.01199162, 0.02302345, -0.07570152,
-0.10080803, 0.01720057, -0.16969536, 0.04589612, -0.02813353],
[-0.18160583, -0.13978417, 0.04127125, 0.16571295, -0.03284559,
0.0764231 , -0.04371225, -0.16360879, 0.01129349, 0.13717636],
[-0.07581151, -0.0351604 , 0.0126438 , -0.00530934, -0.09896594,
-0.04470657, -0.01918913, -0.1446259 , 0.0203516 , -0.01299113],
[-0.09019139, -0.0571584 , -0.00440881, -0.02648136, -0.06586411,
-0.01910152, 0.03425714, -0.0319499 , -0.00051865, 0.10814659],
[-0.06393736, -0.048402 , 0.00892065, 0.02362514, -0.00867739,
-0.06464607, 0.03171837, -0.01439925, 0.02972389, 0.13565545],
[-0.1498428 , -0.04203952, -0.00111851, 0.10854436, -0.09206766,
0.0091841 , 0.03095595, -0.14561233, 0.00472788, 0.08544257],
[-0.20887655, 0.02444044, 0.11485076, -0.02688832, -0.05181421,
0.05912109, 0.02082806, -0.0917733 , -0.07798172, 0.11812017],
[-0.1129831 , -0.02640573, 0.02469945, -0.01530576, -0.1967389 ,
-0.07836114, -0.06786687, -0.19312854, -0.01617849, 0.02968186],
[-0.01551327, 0.06995746, -0.01519905, -0.08725969, -0.08847767,
0.05924645, 0.05522818, -0.06199178, -0.022707 , 0.04745337],
[-0.05959934, -0.07581387, 0.04323972, -0.04583518, -0.06102632,
-0.04827818, 0.03426407, -0.1277703 , -0.00974616, 0.05396844],
[-0.03120111, -0.03422236, 0.09112003, -0.02818224, 0.00751239,
0.04663142, 0.04077137, -0.0319164 , -0.06942125, 0.11782715],
[-0.18401295, -0.10521346, 0.04947029, -0.00101419, -0.18218213,
-0.00622979, -0.07917031, -0.20292684, -0.01392259, 0.0512893 ],
[-0.12129581, 0.0049009 , 0.04994124, -0.09951019, -0.14505789,
0.05412545, 0.00843124, -0.14207928, 0.07050267, -0.00762532],
[-0.06950583, -0.05331536, -0.0125838 , -0.08974443, -0.01576433,
-0.10772426, -0.00632659, -0.03087209, -0.03734441, 0.06967355],
[-0.11330047, -0.03457201, 0.05963513, 0.01188495, 0.0012813 ,
-0.056724 , 0.07425238, -0.05277427, 0.01197292, 0.06704658],
[-0.13196674, -0.02214321, 0.00215278, 0.02572862, -0.18054628,
0.04058442, 0.00691799, -0.2053982 , 0.03001385, 0.04406084],
[-0.17465061, -0.11782445, -0.04236503, 0.08236577, -0.2051768 ,
0.0226867 , -0.03547823, -0.26090765, 0.12756164, 0.09971047],
[-0.10532928, 0.05967205, 0.0923636 , -0.07201086, -0.0242336 ,
-0.0048001 , 0.03947905, -0.11094075, -0.02473236, 0.03018067],
[-0.17736793, -0.16693729, 0.01410204, 0.08446591, -0.14209296,
0.01228525, -0.11250666, -0.20246303, -0.03272526, 0.09422274],
[-0.06190823, -0.01151378, 0.0151554 , -0.01628277, -0.04089763,
0.05539602, 0.06337091, -0.0084438 , 0.03406027, 0.02725619],
[-0.1749413 , 0.03524211, 0.0105392 , -0.07319914, -0.0777961 ,
0.01221193, -0.00199322, -0.04091456, -0.02219761, 0.12453829],
[-0.05857544, -0.0285053 , 0.02454159, -0.00040605, -0.07742713,
0.04547 , 0.06914302, -0.0118324 , 0.10275047, 0.0432786 ],
[-0.14534897, -0.07914923, 0.07591049, -0.05723767, -0.15003572,
0.03480105, 0.06121707, -0.08198671, -0.0590301 , 0.10555428],
[-0.13663602, -0.04123668, 0.05517209, 0.05949097, -0.03141512,
-0.06588402, 0.08407901, -0.06183365, -0.008428 , 0.08748186],
[-0.09815326, 0.05892584, 0.1558336 , -0.00630691, -0.06840624,
0.13607553, 0.0812901 , -0.07638689, -0.03336222, 0.09895931],
[-0.17751525, 0.01755818, 0.03794699, -0.06631032, -0.05646997,
-0.00797169, 0.07238399, -0.03438905, -0.00524065, 0.09366383],
[-0.02225663, -0.05085301, 0.08878558, 0.01996073, -0.08790978,
-0.02905351, -0.06463408, -0.1858761 , 0.05886184, 0.00491168],
[-0.165163 , 0.00737038, 0.05955574, -0.07828765, -0.02529719,
-0.06453862, 0.0340358 , -0.00464464, -0.04846876, 0.10016941],
[-0.02606273, -0.04760323, 0.03779679, 0.04201802, -0.02642431,
-0.10690506, 0.02898183, -0.114568 , 0.05397899, 0.01628222],
[-0.02865287, -0.04724592, 0.03511005, -0.08263997, -0.07223832,
-0.00330405, -0.00054062, -0.05890358, -0.0048471 , 0.04592701],
[-0.17038593, 0.04257239, 0.0932904 , -0.12936944, 0.02024365,
-0.04084445, 0.02122457, 0.00182849, -0.08776811, 0.06233242],
[-0.13186829, -0.03954299, -0.01671081, 0.03870502, -0.19246385,
-0.02992961, -0.03405046, -0.19457611, 0.06994209, 0.05223263]],
dtype=float32)>
} }
W0000 00:00:1793849658.947836 72427 cache_dataset_ops.cc:912] The calling iterator did not fully read the dataset being cached. In order to avoid unexpected truncation of the dataset, the partially cached contents of the dataset will be discarded. This can happen if you have an input pipeline similar to `dataset.cache().take(k).repeat()`. You should use `dataset.take(k).cache().repeat()` instead.
Calling the restored function is just a forward pass on the saved model (tf.keras.Model.predict). What if you want to continue training the loaded function? Or what if you need to embed the loaded function into a bigger model? A common practice is to wrap this loaded object into a Keras layer to achieve this. Luckily, TF Hub has hub.KerasLayer for this purpose, shown here:
import tensorflow_hub as hub
def build_model(loaded):
x = tf.keras.layers.Input(shape=(28, 28, 1), name='input_x')
# Wrap what's loaded to a KerasLayer
keras_layer = hub.KerasLayer(loaded, trainable=True)(x)
model = tf.keras.Model(x, keras_layer)
return model
another_strategy = tf.distribute.MirroredStrategy()
with another_strategy.scope():
loaded = tf.saved_model.load(saved_model_path)
model = build_model(loaded)
model.compile(loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=tf.keras.optimizers.Adam(),
metrics=[tf.metrics.SparseCategoricalAccuracy()])
model.fit(train_dataset, epochs=2)
/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_hub/__init__.py:61: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
from pkg_resources import parse_version
INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0', '/job:localhost/replica:0/task:0/device:GPU:1', '/job:localhost/replica:0/task:0/device:GPU:2', '/job:localhost/replica:0/task:0/device:GPU:3')
INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0', '/job:localhost/replica:0/task:0/device:GPU:1', '/job:localhost/replica:0/task:0/device:GPU:2', '/job:localhost/replica:0/task:0/device:GPU:3')
W0000 00:00:1793849659.522934 72427 auto_shard.cc:559] The `assert_cardinality` transformation is currently not handled by the auto-shard rewrite and will be removed.
Epoch 1/2
INFO:tensorflow:Collective all_reduce tensors: 6 all_reduces, num_devices = 4, group_size = 4, implementation = CommunicationImplementation.NCCL, num_packs = 1
INFO:tensorflow:Collective all_reduce tensors: 6 all_reduces, num_devices = 4, group_size = 4, implementation = CommunicationImplementation.NCCL, num_packs = 1
INFO:tensorflow:Collective all_reduce tensors: 6 all_reduces, num_devices = 4, group_size = 4, implementation = CommunicationImplementation.NCCL, num_packs = 1
INFO:tensorflow:Collective all_reduce tensors: 6 all_reduces, num_devices = 4, group_size = 4, implementation = CommunicationImplementation.NCCL, num_packs = 1
235/235 [==============================] - 5s 7ms/step - loss: 0.3445 - sparse_categorical_accuracy: 0.9040
Epoch 2/2
235/235 [==============================] - 2s 7ms/step - loss: 0.1022 - sparse_categorical_accuracy: 0.9713
In the above example, Tensorflow Hub's hub.KerasLayer wraps the result loaded back from tf.saved_model.load into a Keras layer that is used to build another model. This is very useful for transfer learning.
Which API should I use?
For saving, if you are working with a Keras model, use the Keras Model.save API unless you need the additional control allowed by the low-level API. If what you are saving is not a Keras model, then the lower-level API, tf.saved_model.save, is your only choice.
For loading, your API choice depends on what you want to get from the model loading API. If you cannot (or do not want to) get a Keras model, then use tf.saved_model.load. Otherwise, use tf.keras.models.load_model. Note that you can get a Keras model back only if you saved a Keras model.
It is possible to mix and match the APIs. You can save a Keras model with Model.save, and load a non-Keras model with the low-level API, tf.saved_model.load.
model = get_model()
# Saving the model using Keras `Model.save`
model.save(saved_model_path)
another_strategy = tf.distribute.MirroredStrategy()
# Loading the model using the lower-level API
with another_strategy.scope():
loaded = tf.saved_model.load(saved_model_path)
INFO:tensorflow:Assets written to: /tmp/tf_save/assets
INFO:tensorflow:Assets written to: /tmp/tf_save/assets
INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0', '/job:localhost/replica:0/task:0/device:GPU:1', '/job:localhost/replica:0/task:0/device:GPU:2', '/job:localhost/replica:0/task:0/device:GPU:3')
INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0', '/job:localhost/replica:0/task:0/device:GPU:1', '/job:localhost/replica:0/task:0/device:GPU:2', '/job:localhost/replica:0/task:0/device:GPU:3')
Saving/Loading from a local device
When saving and loading from a local I/O device while training on remote devices—for example, when using a Cloud TPU—you must use the option experimental_io_device in tf.saved_model.SaveOptions and tf.saved_model.LoadOptions to set the I/O device to localhost. For example:
model = get_model()
# Saving the model to a path on localhost.
saved_model_path = '/tmp/tf_save'
save_options = tf.saved_model.SaveOptions(experimental_io_device='/job:localhost')
model.save(saved_model_path, options=save_options)
# Loading the model from a path on localhost.
another_strategy = tf.distribute.MirroredStrategy()
with another_strategy.scope():
load_options = tf.saved_model.LoadOptions(experimental_io_device='/job:localhost')
loaded = tf.keras.models.load_model(saved_model_path, options=load_options)
INFO:tensorflow:Assets written to: /tmp/tf_save/assets
INFO:tensorflow:Assets written to: /tmp/tf_save/assets
INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0', '/job:localhost/replica:0/task:0/device:GPU:1', '/job:localhost/replica:0/task:0/device:GPU:2', '/job:localhost/replica:0/task:0/device:GPU:3')
INFO:tensorflow:Using MirroredStrategy with devices ('/job:localhost/replica:0/task:0/device:GPU:0', '/job:localhost/replica:0/task:0/device:GPU:1', '/job:localhost/replica:0/task:0/device:GPU:2', '/job:localhost/replica:0/task:0/device:GPU:3')
Caveats
One special case is when you create Keras models in certain ways, and then save them before training. For example:
class SubclassedModel(tf.keras.Model):
"""Example model defined by subclassing `tf.keras.Model`."""
output_name = 'output_layer'
def __init__(self):
super(SubclassedModel, self).__init__()
self._dense_layer = tf.keras.layers.Dense(
5, dtype=tf.dtypes.float32, name=self.output_name)
def call(self, inputs):
return self._dense_layer(inputs)
my_model = SubclassedModel()
try:
my_model.save(saved_model_path)
except ValueError as e:
print(f'{type(e).__name__}: ', *e.args)
WARNING:tensorflow:Skipping full serialization of TF-Keras layer <__main__.SubclassedModel object at 0x7f8d4b5206d0>, because it is not built. WARNING:tensorflow:Skipping full serialization of TF-Keras layer <__main__.SubclassedModel object at 0x7f8d4b5206d0>, because it is not built. ValueError: Model <__main__.SubclassedModel object at 0x7f8d4b5206d0> cannot be saved either because the input shape is not available or because the forward pass of the model is not defined.To define a forward pass, please override `Model.call()`. To specify an input shape, either call `build(input_shape)` directly, or call the model on actual data using `Model()`, `Model.fit()`, or `Model.predict()`. If you have a custom training step, please make sure to invoke the forward pass in train step through `Model.__call__`, i.e. `model(inputs)`, as opposed to `model.call()`.
A SavedModel saves the tf.types.experimental.ConcreteFunction objects generated when you trace a tf.function (check When is a Function tracing? in the Introduction to graphs and tf.function guide to learn more). If you get a ValueError like this it's because Model.save was not able to find or create a traced ConcreteFunction.
tf.saved_model.save(my_model, saved_model_path)
x = tf.saved_model.load(saved_model_path)
x.signatures
WARNING:tensorflow:Skipping full serialization of TF-Keras layer <__main__.SubclassedModel object at 0x7f8d4b5206d0>, because it is not built.
WARNING:tensorflow:Skipping full serialization of TF-Keras layer <__main__.SubclassedModel object at 0x7f8d4b5206d0>, because it is not built.
WARNING:tensorflow:Skipping full serialization of TF-Keras layer <tf_keras.src.layers.core.dense.Dense object at 0x7f8d4a911590>, because it is not built.
WARNING:tensorflow:Skipping full serialization of TF-Keras layer <tf_keras.src.layers.core.dense.Dense object at 0x7f8d4a911590>, because it is not built.
INFO:tensorflow:Assets written to: /tmp/tf_save/assets
INFO:tensorflow:Assets written to: /tmp/tf_save/assets
_SignatureMap({})
Usually the model's forward pass—the call method—will be traced automatically when the model is called for the first time, often via the Keras Model.fit method. A ConcreteFunction can also be generated by the Keras Sequential and Functional APIs, if you set the input shape, for example, by making the first layer either a tf.keras.layers.InputLayer or another layer type, and passing it the input_shape keyword argument.
To verify if your model has any traced ConcreteFunctions, check if Model.save_spec is None:
print(my_model.save_spec() is None)
True
Let's use tf.keras.Model.fit to train the model, and notice that the save_spec gets defined and model saving will work:
BATCH_SIZE_PER_REPLICA = 4
BATCH_SIZE = BATCH_SIZE_PER_REPLICA * mirrored_strategy.num_replicas_in_sync
dataset_size = 100
dataset = tf.data.Dataset.from_tensors(
(tf.range(5, dtype=tf.float32), tf.range(5, dtype=tf.float32))
).repeat(dataset_size).batch(BATCH_SIZE)
my_model.compile(optimizer='adam', loss='mean_squared_error')
my_model.fit(dataset, epochs=2)
print(my_model.save_spec() is None)
my_model.save(saved_model_path)
Epoch 1/2
7/7 [==============================] - 1s 3ms/step - loss: 8.0419
Epoch 2/2
7/7 [==============================] - 0s 2ms/step - loss: 7.6664
False
INFO:tensorflow:Unsupported signature for serialization: ((TensorSpec(shape=(5, 5), dtype=tf.float32, name='gradient'), <tensorflow.python.framework.func_graph.UnknownArgument object at 0x7f8d4ab3f810>, 140244820477584), {}).
INFO:tensorflow:Unsupported signature for serialization: ((TensorSpec(shape=(5, 5), dtype=tf.float32, name='gradient'), <tensorflow.python.framework.func_graph.UnknownArgument object at 0x7f8d4ab3f810>, 140244820477584), {}).
INFO:tensorflow:Unsupported signature for serialization: ((TensorSpec(shape=(5,), dtype=tf.float32, name='gradient'), <tensorflow.python.framework.func_graph.UnknownArgument object at 0x7f8d4aacb250>, 140244820482672), {}).
INFO:tensorflow:Unsupported signature for serialization: ((TensorSpec(shape=(5,), dtype=tf.float32, name='gradient'), <tensorflow.python.framework.func_graph.UnknownArgument object at 0x7f8d4aacb250>, 140244820482672), {}).
INFO:tensorflow:Unsupported signature for serialization: ((TensorSpec(shape=(5, 5), dtype=tf.float32, name='gradient'), <tensorflow.python.framework.func_graph.UnknownArgument object at 0x7f8d4ab3f810>, 140244820477584), {}).
INFO:tensorflow:Unsupported signature for serialization: ((TensorSpec(shape=(5, 5), dtype=tf.float32, name='gradient'), <tensorflow.python.framework.func_graph.UnknownArgument object at 0x7f8d4ab3f810>, 140244820477584), {}).
INFO:tensorflow:Unsupported signature for serialization: ((TensorSpec(shape=(5,), dtype=tf.float32, name='gradient'), <tensorflow.python.framework.func_graph.UnknownArgument object at 0x7f8d4aacb250>, 140244820482672), {}).
INFO:tensorflow:Unsupported signature for serialization: ((TensorSpec(shape=(5,), dtype=tf.float32, name='gradient'), <tensorflow.python.framework.func_graph.UnknownArgument object at 0x7f8d4aacb250>, 140244820482672), {}).
INFO:tensorflow:Assets written to: /tmp/tf_save/assets
INFO:tensorflow:Assets written to: /tmp/tf_save/assets