 Ver en TensorFlow.org |  Ejecutar en Google Colab |  Ver fuente en GitHub |  Descargar cuaderno |
Bienvenido a la guía completa para la poda de peso Keras.
Esta página documenta varios casos de uso y muestra cómo utilizar la API para cada uno. Una vez que sepa qué API necesita, busque los parámetros y los detalles de bajo nivel en los documentos de API .
- Si desea ver los beneficios de la poda y lo que se admite, consulte la descripción general .
- Para ver un solo ejemplo de extremo a extremo, consulte el ejemplo de poda .
Se cubren los siguientes casos de uso:
- Definir y entrenar un modelo podado.
- Secuencial y funcional.
- Bucles de entrenamiento Keras model.fit y custom
- Controle y deserialice un modelo podado.
- Implemente un modelo podado y vea los beneficios de la compresión.
Para configurar el algoritmo de poda, consulte los documentos de la API tfmot.sparsity.keras.prune_low_magnitude .
Preparar
Para encontrar las API que necesita y comprender los propósitos, puede ejecutar pero omitir la lectura de esta sección.
! pip install -q tensorflow-model-optimization
import tensorflow as tf
import numpy as np
import tensorflow_model_optimization as tfmot
%load_ext tensorboard
import tempfile
input_shape = [20]
x_train = np.random.randn(1, 20).astype(np.float32)
y_train = tf.keras.utils.to_categorical(np.random.randn(1), num_classes=20)
def setup_model():
model = tf.keras.Sequential([
tf.keras.layers.Dense(20, input_shape=input_shape),
tf.keras.layers.Flatten()
])
return model
def setup_pretrained_weights():
model = setup_model()
model.compile(
loss=tf.keras.losses.categorical_crossentropy,
optimizer='adam',
metrics=['accuracy']
)
model.fit(x_train, y_train)
_, pretrained_weights = tempfile.mkstemp('.tf')
model.save_weights(pretrained_weights)
return pretrained_weights
def get_gzipped_model_size(model):
# Returns size of gzipped model, in bytes.
import os
import zipfile
_, keras_file = tempfile.mkstemp('.h5')
model.save(keras_file, include_optimizer=False)
_, zipped_file = tempfile.mkstemp('.zip')
with zipfile.ZipFile(zipped_file, 'w', compression=zipfile.ZIP_DEFLATED) as f:
f.write(keras_file)
return os.path.getsize(zipped_file)
setup_model()
pretrained_weights = setup_pretrained_weights()
WARNING: You are using pip version 20.2.2; however, version 20.2.3 is available. You should consider upgrading via the '/tmpfs/src/tf_docs_env/bin/python -m pip install --upgrade pip' command. 1/1 [==============================] - 0s 2ms/step - loss: 1.1999 - accuracy: 0.0000e+00
Definir modelo
Pode el modelo completo (secuencial y funcional)
Consejos para una mejor precisión del modelo:
- Intente "Podar algunas capas" para omitir la poda de las capas que más reducen la precisión.
- Por lo general, es mejor afinar la poda en lugar de entrenar desde cero.
Para hacer que el modelo completo entrene con poda, aplique tfmot.sparsity.keras.prune_low_magnitude al modelo.
base_model = setup_model()
base_model.load_weights(pretrained_weights) # optional but recommended.
model_for_pruning = tfmot.sparsity.keras.prune_low_magnitude(base_model)
model_for_pruning.summary()
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow_model_optimization/python/core/sparsity/keras/pruning_wrapper.py:200: Layer.add_variable (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version. Instructions for updating: Please use `layer.add_weight` method instead. Model: "sequential_2" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= prune_low_magnitude_dense_2 (None, 20) 822 _________________________________________________________________ prune_low_magnitude_flatten_ (None, 20) 1 ================================================================= Total params: 823 Trainable params: 420 Non-trainable params: 403 _________________________________________________________________
Poda algunas capas (secuencial y funcional)
Podar un modelo puede tener un efecto negativo en la precisión. Puede podar capas de un modelo de manera selectiva para explorar el equilibrio entre precisión, velocidad y tamaño del modelo.
Consejos para una mejor precisión del modelo:
- Por lo general, es mejor afinar la poda en lugar de entrenar desde cero.
- Intente podar las últimas capas en lugar de las primeras.
- Evite podar capas críticas (por ejemplo, mecanismo de atención).
Más :
- Los documentos de la API
tfmot.sparsity.keras.prune_low_magnitudeproporcionan detalles sobre cómo variar la configuración de poda por capa.
En el siguiente ejemplo, pode solo las capas Dense .
# Create a base model
base_model = setup_model()
base_model.load_weights(pretrained_weights) # optional but recommended for model accuracy
# Helper function uses `prune_low_magnitude` to make only the
# Dense layers train with pruning.
def apply_pruning_to_dense(layer):
if isinstance(layer, tf.keras.layers.Dense):
return tfmot.sparsity.keras.prune_low_magnitude(layer)
return layer
# Use `tf.keras.models.clone_model` to apply `apply_pruning_to_dense`
# to the layers of the model.
model_for_pruning = tf.keras.models.clone_model(
base_model,
clone_function=apply_pruning_to_dense,
)
model_for_pruning.summary()
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.bias WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details. Model: "sequential_3" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= prune_low_magnitude_dense_3 (None, 20) 822 _________________________________________________________________ flatten_3 (Flatten) (None, 20) 0 ================================================================= Total params: 822 Trainable params: 420 Non-trainable params: 402 _________________________________________________________________
Si bien este ejemplo usó el tipo de capa para decidir qué podar, la forma más fácil de podar una capa en particular es establecer su propiedad de name y buscar ese nombre en clone_function .
print(base_model.layers[0].name)
dense_3
Precisión de modelo más legible pero potencialmente más baja
Esto no es compatible con el ajuste fino con la poda, por lo que puede ser menos preciso que los ejemplos anteriores que admiten el ajuste fino.
Si bien prune_low_magnitude se puede aplicar al definir el modelo inicial, cargar los pesos después no funciona en los ejemplos siguientes.
Ejemplo funcional
# Use `prune_low_magnitude` to make the `Dense` layer train with pruning.
i = tf.keras.Input(shape=(20,))
x = tfmot.sparsity.keras.prune_low_magnitude(tf.keras.layers.Dense(10))(i)
o = tf.keras.layers.Flatten()(x)
model_for_pruning = tf.keras.Model(inputs=i, outputs=o)
model_for_pruning.summary()
Model: "functional_1" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_1 (InputLayer) [(None, 20)] 0 _________________________________________________________________ prune_low_magnitude_dense_4 (None, 10) 412 _________________________________________________________________ flatten_4 (Flatten) (None, 10) 0 ================================================================= Total params: 412 Trainable params: 210 Non-trainable params: 202 _________________________________________________________________
Ejemplo secuencial
# Use `prune_low_magnitude` to make the `Dense` layer train with pruning.
model_for_pruning = tf.keras.Sequential([
tfmot.sparsity.keras.prune_low_magnitude(tf.keras.layers.Dense(20, input_shape=input_shape)),
tf.keras.layers.Flatten()
])
model_for_pruning.summary()
Model: "sequential_4" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= prune_low_magnitude_dense_5 (None, 20) 822 _________________________________________________________________ flatten_5 (Flatten) (None, 20) 0 ================================================================= Total params: 822 Trainable params: 420 Non-trainable params: 402 _________________________________________________________________
Pode la capa personalizada de Keras o modifique partes de la capa para podar
Error común: recortar el sesgo suele dañar demasiado la precisión del modelo.
tfmot.sparsity.keras.PrunableLayer sirve para dos casos de uso:
- Poda una capa personalizada de Keras
- Modifique partes de una capa de Keras incorporada para podar.
Por ejemplo, la API de forma predeterminada solo poda el kernel de la capa Dense . El siguiente ejemplo también elimina el sesgo.
class MyDenseLayer(tf.keras.layers.Dense, tfmot.sparsity.keras.PrunableLayer):
def get_prunable_weights(self):
# Prune bias also, though that usually harms model accuracy too much.
return [self.kernel, self.bias]
# Use `prune_low_magnitude` to make the `MyDenseLayer` layer train with pruning.
model_for_pruning = tf.keras.Sequential([
tfmot.sparsity.keras.prune_low_magnitude(MyDenseLayer(20, input_shape=input_shape)),
tf.keras.layers.Flatten()
])
model_for_pruning.summary()
Model: "sequential_5" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= prune_low_magnitude_my_dense (None, 20) 843 _________________________________________________________________ flatten_6 (Flatten) (None, 20) 0 ================================================================= Total params: 843 Trainable params: 420 Non-trainable params: 423 _________________________________________________________________
Modelo de tren
Model.fit
Llame a la tfmot.sparsity.keras.UpdatePruningStep llamada tfmot.sparsity.keras.UpdatePruningStep durante el entrenamiento.
Para ayudar a depurar el entrenamiento, use la tfmot.sparsity.keras.PruningSummaries llamada tfmot.sparsity.keras.PruningSummaries .
# Define the model.
base_model = setup_model()
base_model.load_weights(pretrained_weights) # optional but recommended for model accuracy
model_for_pruning = tfmot.sparsity.keras.prune_low_magnitude(base_model)
log_dir = tempfile.mkdtemp()
callbacks = [
tfmot.sparsity.keras.UpdatePruningStep(),
# Log sparsity and other metrics in Tensorboard.
tfmot.sparsity.keras.PruningSummaries(log_dir=log_dir)
]
model_for_pruning.compile(
loss=tf.keras.losses.categorical_crossentropy,
optimizer='adam',
metrics=['accuracy']
)
model_for_pruning.fit(
x_train,
y_train,
callbacks=callbacks,
epochs=2,
)
%tensorboard --logdir={log_dir}
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.bias WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details. Epoch 1/2 1/1 [==============================] - 0s 3ms/step - loss: 1.2485 - accuracy: 0.0000e+00 Epoch 2/2 WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/ops/summary_ops_v2.py:1277: stop (from tensorflow.python.eager.profiler) is deprecated and will be removed after 2020-07-01. Instructions for updating: use `tf.profiler.experimental.stop` instead. 1/1 [==============================] - 0s 2ms/step - loss: 1.1999 - accuracy: 0.0000e+00
Para los usuarios que no son de Colab, puede ver los resultados de una ejecución anterior de este bloque de código en TensorBoard.dev .
Bucle de entrenamiento personalizado
Llame a la tfmot.sparsity.keras.UpdatePruningStep llamada tfmot.sparsity.keras.UpdatePruningStep durante el entrenamiento.
Para ayudar a depurar el entrenamiento, use la tfmot.sparsity.keras.PruningSummaries llamada tfmot.sparsity.keras.PruningSummaries .
# Define the model.
base_model = setup_model()
base_model.load_weights(pretrained_weights) # optional but recommended for model accuracy
model_for_pruning = tfmot.sparsity.keras.prune_low_magnitude(base_model)
# Boilerplate
loss = tf.keras.losses.categorical_crossentropy
optimizer = tf.keras.optimizers.Adam()
log_dir = tempfile.mkdtemp()
unused_arg = -1
epochs = 2
batches = 1 # example is hardcoded so that the number of batches cannot change.
# Non-boilerplate.
model_for_pruning.optimizer = optimizer
step_callback = tfmot.sparsity.keras.UpdatePruningStep()
step_callback.set_model(model_for_pruning)
log_callback = tfmot.sparsity.keras.PruningSummaries(log_dir=log_dir) # Log sparsity and other metrics in Tensorboard.
log_callback.set_model(model_for_pruning)
step_callback.on_train_begin() # run pruning callback
for _ in range(epochs):
log_callback.on_epoch_begin(epoch=unused_arg) # run pruning callback
for _ in range(batches):
step_callback.on_train_batch_begin(batch=unused_arg) # run pruning callback
with tf.GradientTape() as tape:
logits = model_for_pruning(x_train, training=True)
loss_value = loss(y_train, logits)
grads = tape.gradient(loss_value, model_for_pruning.trainable_variables)
optimizer.apply_gradients(zip(grads, model_for_pruning.trainable_variables))
step_callback.on_epoch_end(batch=unused_arg) # run pruning callback
%tensorboard --logdir={log_dir}
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.bias WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.
Para los usuarios que no son de Colab, puede ver los resultados de una ejecución anterior de este bloque de código en TensorBoard.dev .
Mejorar la precisión del modelo podado
Primero, mire los documentos de la API tfmot.sparsity.keras.prune_low_magnitude para comprender qué es un programa de poda y las matemáticas de cada tipo de programa de poda.
Consejos :
Tenga una tasa de aprendizaje que no sea ni demasiado alta ni demasiado baja cuando el modelo esté podando. Considere el programa de poda como un hiperparámetro.
Como prueba rápida, intente experimentar con la poda de un modelo a la escasez final al comienzo del entrenamiento estableciendo
begin_stepen 0 con un programatfmot.sparsity.keras.ConstantSparsity. Puede tener suerte con buenos resultados.No pode con mucha frecuencia para que el modelo tenga tiempo de recuperarse. El programa de poda proporciona una frecuencia predeterminada decente.
Para obtener ideas generales para mejorar la precisión del modelo, busque sugerencias para su (s) caso (s) de uso en "Definir modelo".
Punto de control y deserialización
Debe conservar el paso del optimizador durante los puntos de control. Esto significa que si bien puede usar los modelos Keras HDF5 para puntos de control, no puede usar pesos Keras HDF5.
# Define the model.
base_model = setup_model()
base_model.load_weights(pretrained_weights) # optional but recommended for model accuracy
model_for_pruning = tfmot.sparsity.keras.prune_low_magnitude(base_model)
_, keras_model_file = tempfile.mkstemp('.h5')
# Checkpoint: saving the optimizer is necessary (include_optimizer=True is the default).
model_for_pruning.save(keras_model_file, include_optimizer=True)
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.bias WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.
Lo anterior se aplica generalmente. El código siguiente solo es necesario para el formato del modelo HDF5 (no los pesos HDF5 y otros formatos).
# Deserialize model.
with tfmot.sparsity.keras.prune_scope():
loaded_model = tf.keras.models.load_model(keras_model_file)
loaded_model.summary()
WARNING:tensorflow:No training configuration found in the save file, so the model was *not* compiled. Compile it manually. Model: "sequential_8" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= prune_low_magnitude_dense_8 (None, 20) 822 _________________________________________________________________ prune_low_magnitude_flatten_ (None, 20) 1 ================================================================= Total params: 823 Trainable params: 420 Non-trainable params: 403 _________________________________________________________________
Implementar modelo podado
Exportar modelo con compresión de tamaño
Error común : tanto strip_pruning como aplicar un algoritmo de compresión estándar (por ejemplo, a través de gzip) son necesarios para ver los beneficios de la poda de compresión.
# Define the model.
base_model = setup_model()
base_model.load_weights(pretrained_weights) # optional but recommended for model accuracy
model_for_pruning = tfmot.sparsity.keras.prune_low_magnitude(base_model)
# Typically you train the model here.
model_for_export = tfmot.sparsity.keras.strip_pruning(model_for_pruning)
print("final model")
model_for_export.summary()
print("\n")
print("Size of gzipped pruned model without stripping: %.2f bytes" % (get_gzipped_model_size(model_for_pruning)))
print("Size of gzipped pruned model with stripping: %.2f bytes" % (get_gzipped_model_size(model_for_export)))
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.bias WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details. final model Model: "sequential_9" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= dense_9 (Dense) (None, 20) 420 _________________________________________________________________ flatten_10 (Flatten) (None, 20) 0 ================================================================= Total params: 420 Trainable params: 420 Non-trainable params: 0 _________________________________________________________________ Size of gzipped pruned model without stripping: 3299.00 bytes Size of gzipped pruned model with stripping: 2876.00 bytes
Optimizaciones específicas de hardware
Una vez que los diferentes backends permiten la poda para mejorar la latencia , el uso de la escasez de bloques puede mejorar la latencia para cierto hardware.
El aumento del tamaño del bloque disminuirá la dispersión máxima que se puede lograr para la precisión del modelo de destino. A pesar de esto, la latencia aún puede mejorar.
Para obtener detalles sobre lo que se admite para la dispersión de bloques, consulte los documentos de la API tfmot.sparsity.keras.prune_low_magnitude .
base_model = setup_model()
# For using intrinsics on a CPU with 128-bit registers, together with 8-bit
# quantized weights, a 1x16 block size is nice because the block perfectly
# fits into the register.
pruning_params = {'block_size': [1, 16]}
model_for_pruning = tfmot.sparsity.keras.prune_low_magnitude(base_model, **pruning_params)
model_for_pruning.summary()
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.bias WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details. Model: "sequential_10" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= prune_low_magnitude_dense_10 (None, 20) 822 _________________________________________________________________ prune_low_magnitude_flatten_ (None, 20) 1 ================================================================= Total params: 823 Trainable params: 420 Non-trainable params: 403 _________________________________________________________________