![]() |
![]() |
![]() |
![]() |
Overview
TensorFlow Estimators are supported in TensorFlow, and can be created from new and existing tf.keras
models. This tutorial contains a complete, minimal example of that process.
Setup
import tensorflow as tf
import numpy as np
import tensorflow_datasets as tfds
2022-12-14 04:25:34.830633: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory 2022-12-14 04:25:34.830733: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory 2022-12-14 04:25:34.830742: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
Create a simple Keras model.
In Keras, you assemble layers to build models. A model is (usually) a graph
of layers. The most common type of model is a stack of layers: the
tf.keras.Sequential
model.
To build a simple, fully-connected network (i.e. multi-layer perceptron):
model = tf.keras.models.Sequential([
tf.keras.layers.Dense(16, activation='relu', input_shape=(4,)),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(3)
])
Compile the model and get a summary.
model.compile(loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer='adam')
model.summary()
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= dense (Dense) (None, 16) 80 dropout (Dropout) (None, 16) 0 dense_1 (Dense) (None, 3) 51 ================================================================= Total params: 131 Trainable params: 131 Non-trainable params: 0 _________________________________________________________________
Create an input function
Use the Datasets API to scale to large datasets or multi-device training.
Estimators need control of when and how their input pipeline is built. To allow this, they require an "Input function" or input_fn
. The Estimator
will call this function with no arguments. The input_fn
must return a tf.data.Dataset
.
def input_fn():
split = tfds.Split.TRAIN
dataset = tfds.load('iris', split=split, as_supervised=True)
dataset = dataset.map(lambda features, labels: ({'dense_input':features}, labels))
dataset = dataset.batch(32).repeat()
return dataset
Test out your input_fn
for features_batch, labels_batch in input_fn().take(1):
print(features_batch)
print(labels_batch)
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/autograph/pyct/static_analysis/liveness.py:83: Analyzer.lamba_check (from tensorflow.python.autograph.pyct.static_analysis.liveness) is deprecated and will be removed after 2023-09-23. Instructions for updating: Lambda fuctions will be no more assumed to be used in the statement where they are used, or at least in the same block. https://github.com/tensorflow/tensorflow/issues/56089 WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/autograph/pyct/static_analysis/liveness.py:83: Analyzer.lamba_check (from tensorflow.python.autograph.pyct.static_analysis.liveness) is deprecated and will be removed after 2023-09-23. Instructions for updating: Lambda fuctions will be no more assumed to be used in the statement where they are used, or at least in the same block. https://github.com/tensorflow/tensorflow/issues/56089 {'dense_input': <tf.Tensor: shape=(32, 4), dtype=float32, numpy= array([[5.1, 3.4, 1.5, 0.2], [7.7, 3. , 6.1, 2.3], [5.7, 2.8, 4.5, 1.3], [6.8, 3.2, 5.9, 2.3], [5.2, 3.4, 1.4, 0.2], [5.6, 2.9, 3.6, 1.3], [5.5, 2.6, 4.4, 1.2], [5.5, 2.4, 3.7, 1. ], [4.6, 3.4, 1.4, 0.3], [7.7, 2.8, 6.7, 2. ], [7. , 3.2, 4.7, 1.4], [4.6, 3.2, 1.4, 0.2], [6.5, 3. , 5.2, 2. ], [5.5, 4.2, 1.4, 0.2], [5.4, 3.9, 1.3, 0.4], [5. , 3.5, 1.3, 0.3], [5.1, 3.8, 1.5, 0.3], [4.8, 3. , 1.4, 0.1], [6.5, 3. , 5.8, 2.2], [7.6, 3. , 6.6, 2.1], [6.7, 3.3, 5.7, 2.1], [7.9, 3.8, 6.4, 2. ], [6.7, 3. , 5.2, 2.3], [5.8, 4. , 1.2, 0.2], [6.3, 2.5, 5. , 1.9], [5. , 3. , 1.6, 0.2], [6.9, 3.1, 5.1, 2.3], [6.1, 3. , 4.6, 1.4], [5.8, 2.7, 4.1, 1. ], [5.2, 2.7, 3.9, 1.4], [6.7, 3. , 5. , 1.7], [5.7, 2.6, 3.5, 1. ]], dtype=float32)>} tf.Tensor([0 2 1 2 0 1 1 1 0 2 1 0 2 0 0 0 0 0 2 2 2 2 2 0 2 0 2 1 1 1 1 1], shape=(32,), dtype=int64)
Create an Estimator from the tf.keras model.
A tf.keras.Model
can be trained with the tf.estimator
API by converting the
model to an tf.estimator.Estimator
object with
tf.keras.estimator.model_to_estimator
.
import tempfile
model_dir = tempfile.mkdtemp()
keras_estimator = tf.keras.estimator.model_to_estimator(
keras_model=model, model_dir=model_dir)
INFO:tensorflow:Using default config. INFO:tensorflow:Using default config. INFO:tensorflow:Using the Keras model provided. INFO:tensorflow:Using the Keras model provided. WARNING:absl:You are using `tf.keras.optimizers.experimental.Optimizer` in TF estimator, which only supports `tf.keras.optimizers.legacy.Optimizer`. Automatically converting your optimizer to `tf.keras.optimizers.legacy.Optimizer`. /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/keras/backend.py:451: UserWarning: `tf.keras.backend.set_learning_phase` is deprecated and will be removed after 2020-10-11. To update it, simply pass a True/False value to the `training` argument of the `__call__` method of your layer or model. warnings.warn( 2022-12-14 04:25:41.135354: W tensorflow/c/c_api.cc:291] Operation '{name:'training/Adam/dense_1/kernel/v/Assign' id:213 op device:{requested: '', assigned: ''} def:{ { {node training/Adam/dense_1/kernel/v/Assign} } = AssignVariableOp[_has_manual_control_dependencies=true, dtype=DT_FLOAT, validate_shape=false](training/Adam/dense_1/kernel/v, training/Adam/dense_1/kernel/v/Initializer/zeros)} }' was changed by setting attribute after it was run by a session. This mutation will have no effect, and will trigger an error in the future. Either don't modify nodes after running them or create a new session. INFO:tensorflow:Using config: {'_model_dir': '/tmpfs/tmp/tmp8f0yqzpt', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true graph_options { rewrite_options { meta_optimizer_iterations: ONE } } , '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1} INFO:tensorflow:Using config: {'_model_dir': '/tmpfs/tmp/tmp8f0yqzpt', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': allow_soft_placement: true graph_options { rewrite_options { meta_optimizer_iterations: ONE } } , '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_train_distribute': None, '_device_fn': None, '_protocol': None, '_eval_distribute': None, '_experimental_distribute': None, '_experimental_max_worker_delay_secs': None, '_session_creation_timeout_secs': 7200, '_checkpoint_save_graph_def': True, '_service': None, '_cluster_spec': ClusterSpec({}), '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}
Train and evaluate the estimator.
keras_estimator.train(input_fn=input_fn, steps=500)
eval_result = keras_estimator.evaluate(input_fn=input_fn, steps=10)
print('Eval result: {}'.format(eval_result))
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/training/training_util.py:396: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version. Instructions for updating: Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts. WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/training/training_util.py:396: Variable.initialized_value (from tensorflow.python.ops.variables) is deprecated and will be removed in a future version. Instructions for updating: Use Variable.read_value. Variables in 2.X are initialized automatically both in eager and graph (inside tf.defun) contexts. INFO:tensorflow:Calling model_fn. INFO:tensorflow:Calling model_fn. INFO:tensorflow:Done calling model_fn. INFO:tensorflow:Done calling model_fn. INFO:tensorflow:Warm-starting with WarmStartSettings: WarmStartSettings(ckpt_to_initialize_from='/tmpfs/tmp/tmp8f0yqzpt/keras/keras_model.ckpt', vars_to_warm_start='.*', var_name_to_vocab_info={}, var_name_to_prev_var_name={}) INFO:tensorflow:Warm-starting with WarmStartSettings: WarmStartSettings(ckpt_to_initialize_from='/tmpfs/tmp/tmp8f0yqzpt/keras/keras_model.ckpt', vars_to_warm_start='.*', var_name_to_vocab_info={}, var_name_to_prev_var_name={}) INFO:tensorflow:Warm-starting from: /tmpfs/tmp/tmp8f0yqzpt/keras/keras_model.ckpt INFO:tensorflow:Warm-starting from: /tmpfs/tmp/tmp8f0yqzpt/keras/keras_model.ckpt INFO:tensorflow:Warm-starting variables only in TRAINABLE_VARIABLES. INFO:tensorflow:Warm-starting variables only in TRAINABLE_VARIABLES. INFO:tensorflow:Warm-started 4 variables. INFO:tensorflow:Warm-started 4 variables. INFO:tensorflow:Create CheckpointSaverHook. INFO:tensorflow:Create CheckpointSaverHook. INFO:tensorflow:Graph was finalized. INFO:tensorflow:Graph was finalized. INFO:tensorflow:Running local_init_op. INFO:tensorflow:Running local_init_op. INFO:tensorflow:Done running local_init_op. INFO:tensorflow:Done running local_init_op. INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0... INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0... INFO:tensorflow:Saving checkpoints for 0 into /tmpfs/tmp/tmp8f0yqzpt/model.ckpt. INFO:tensorflow:Saving checkpoints for 0 into /tmpfs/tmp/tmp8f0yqzpt/model.ckpt. INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0... INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0... INFO:tensorflow:loss = 3.283334, step = 0 INFO:tensorflow:loss = 3.283334, step = 0 INFO:tensorflow:global_step/sec: 49.0792 INFO:tensorflow:global_step/sec: 49.0792 INFO:tensorflow:loss = 1.3099126, step = 100 (2.039 sec) INFO:tensorflow:loss = 1.3099126, step = 100 (2.039 sec) INFO:tensorflow:global_step/sec: 28.2472 INFO:tensorflow:global_step/sec: 28.2472 INFO:tensorflow:loss = 0.69173384, step = 200 (3.540 sec) INFO:tensorflow:loss = 0.69173384, step = 200 (3.540 sec) INFO:tensorflow:global_step/sec: 50.4165 INFO:tensorflow:global_step/sec: 50.4165 INFO:tensorflow:loss = 0.47193998, step = 300 (1.984 sec) INFO:tensorflow:loss = 0.47193998, step = 300 (1.984 sec) INFO:tensorflow:global_step/sec: 49.1752 INFO:tensorflow:global_step/sec: 49.1752 INFO:tensorflow:loss = 0.4255399, step = 400 (2.033 sec) INFO:tensorflow:loss = 0.4255399, step = 400 (2.033 sec) INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 500... INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 500... INFO:tensorflow:Saving checkpoints for 500 into /tmpfs/tmp/tmp8f0yqzpt/model.ckpt. INFO:tensorflow:Saving checkpoints for 500 into /tmpfs/tmp/tmp8f0yqzpt/model.ckpt. INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 500... INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 500... INFO:tensorflow:Loss for final step: 0.44894162. INFO:tensorflow:Loss for final step: 0.44894162. INFO:tensorflow:Calling model_fn. INFO:tensorflow:Calling model_fn. INFO:tensorflow:Done calling model_fn. /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/keras/engine/training_v1.py:2333: UserWarning: `Model.state_updates` will be removed in a future version. This property should not be used in TensorFlow 2.0, as `updates` are applied automatically. updates = self.state_updates INFO:tensorflow:Done calling model_fn. INFO:tensorflow:Starting evaluation at 2022-12-14T04:25:55 INFO:tensorflow:Starting evaluation at 2022-12-14T04:25:55 INFO:tensorflow:Graph was finalized. INFO:tensorflow:Graph was finalized. INFO:tensorflow:Restoring parameters from /tmpfs/tmp/tmp8f0yqzpt/model.ckpt-500 INFO:tensorflow:Restoring parameters from /tmpfs/tmp/tmp8f0yqzpt/model.ckpt-500 INFO:tensorflow:Running local_init_op. INFO:tensorflow:Running local_init_op. INFO:tensorflow:Done running local_init_op. INFO:tensorflow:Done running local_init_op. INFO:tensorflow:Evaluation [1/10] INFO:tensorflow:Evaluation [1/10] INFO:tensorflow:Evaluation [2/10] INFO:tensorflow:Evaluation [2/10] INFO:tensorflow:Evaluation [3/10] INFO:tensorflow:Evaluation [3/10] INFO:tensorflow:Evaluation [4/10] INFO:tensorflow:Evaluation [4/10] INFO:tensorflow:Evaluation [5/10] INFO:tensorflow:Evaluation [5/10] INFO:tensorflow:Evaluation [6/10] INFO:tensorflow:Evaluation [6/10] INFO:tensorflow:Evaluation [7/10] INFO:tensorflow:Evaluation [7/10] INFO:tensorflow:Evaluation [8/10] INFO:tensorflow:Evaluation [8/10] INFO:tensorflow:Evaluation [9/10] INFO:tensorflow:Evaluation [9/10] INFO:tensorflow:Evaluation [10/10] INFO:tensorflow:Evaluation [10/10] INFO:tensorflow:Inference Time : 0.49018s INFO:tensorflow:Inference Time : 0.49018s INFO:tensorflow:Finished evaluation at 2022-12-14-04:25:55 INFO:tensorflow:Finished evaluation at 2022-12-14-04:25:55 INFO:tensorflow:Saving dict for global step 500: global_step = 500, loss = 0.34172997 INFO:tensorflow:Saving dict for global step 500: global_step = 500, loss = 0.34172997 INFO:tensorflow:Saving 'checkpoint_path' summary for global step 500: /tmpfs/tmp/tmp8f0yqzpt/model.ckpt-500 INFO:tensorflow:Saving 'checkpoint_path' summary for global step 500: /tmpfs/tmp/tmp8f0yqzpt/model.ckpt-500 Eval result: {'loss': 0.34172997, 'global_step': 500}