 Ver en TensorFlow.org |  Ejecutar en Google Colab |  Ver fuente en GitHub |  Descargar cuaderno |
Bienvenido a la guía completa para la poda de peso Keras.
Esta página documenta varios casos de uso y muestra cómo utilizar la API para cada uno. Una vez que sepas lo que las API que necesita, encontrar los parámetros y los detalles de bajo nivel en los documentos de la API .
- Si desea ver los beneficios de la poda y lo que está apoyado, ver el panorama general .
- Para un solo ejemplo de extremo a extremo, ver el ejemplo poda .
Se cubren los siguientes casos de uso:
- Definir y entrenar un modelo podado.
- Secuencial y Funcional.
- Bucles de entrenamiento Keras model.fit y custom
- Controle y deserialice un modelo podado.
- Implemente un modelo podado y vea los beneficios de la compresión.
Para la configuración del algoritmo de poda, consulte las tfmot.sparsity.keras.prune_low_magnitude documentos de la API.
Configuración
Para encontrar las API que necesita y comprender los propósitos, puede ejecutar, pero omita la lectura de esta sección.
! pip install -q tensorflow-model-optimization
import tensorflow as tf
import numpy as np
import tensorflow_model_optimization as tfmot
%load_ext tensorboard
import tempfile
input_shape = [20]
x_train = np.random.randn(1, 20).astype(np.float32)
y_train = tf.keras.utils.to_categorical(np.random.randn(1), num_classes=20)
def setup_model():
model = tf.keras.Sequential([
tf.keras.layers.Dense(20, input_shape=input_shape),
tf.keras.layers.Flatten()
])
return model
def setup_pretrained_weights():
model = setup_model()
model.compile(
loss=tf.keras.losses.categorical_crossentropy,
optimizer='adam',
metrics=['accuracy']
)
model.fit(x_train, y_train)
_, pretrained_weights = tempfile.mkstemp('.tf')
model.save_weights(pretrained_weights)
return pretrained_weights
def get_gzipped_model_size(model):
# Returns size of gzipped model, in bytes.
import os
import zipfile
_, keras_file = tempfile.mkstemp('.h5')
model.save(keras_file, include_optimizer=False)
_, zipped_file = tempfile.mkstemp('.zip')
with zipfile.ZipFile(zipped_file, 'w', compression=zipfile.ZIP_DEFLATED) as f:
f.write(keras_file)
return os.path.getsize(zipped_file)
setup_model()
pretrained_weights = setup_pretrained_weights()
Definir modelo
Pode el modelo completo (secuencial y funcional)
Consejos para una mejor precisión del modelo:
- Intente "Podar algunas capas" para omitir la poda de las capas que más reducen la precisión.
- En general, es mejor afinar con la poda en lugar de entrenar desde cero.
Para que todo el tren modelo con la poda, aplicar tfmot.sparsity.keras.prune_low_magnitude al modelo.
base_model = setup_model()
base_model.load_weights(pretrained_weights) # optional but recommended.
model_for_pruning = tfmot.sparsity.keras.prune_low_magnitude(base_model)
model_for_pruning.summary()
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow_model_optimization/python/core/sparsity/keras/pruning_wrapper.py:200: Layer.add_variable (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version. Instructions for updating: Please use `layer.add_weight` method instead. Model: "sequential_2" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= prune_low_magnitude_dense_2 (None, 20) 822 _________________________________________________________________ prune_low_magnitude_flatten_ (None, 20) 1 ================================================================= Total params: 823 Trainable params: 420 Non-trainable params: 403 _________________________________________________________________
Poda algunas capas (secuencial y funcional)
Podar un modelo puede tener un efecto negativo en la precisión. Puede podar selectivamente capas de un modelo para explorar el equilibrio entre precisión, velocidad y tamaño del modelo.
Consejos para una mejor precisión del modelo:
- En general, es mejor afinar con la poda en lugar de entrenar desde cero.
- Intente podar las últimas capas en lugar de las primeras.
- Evite podar capas críticas (por ejemplo, mecanismo de atención).
más:
- Los
tfmot.sparsity.keras.prune_low_magnitudedocs API proporcionan detalles sobre cómo variar la configuración de poda por capa.
En el siguiente ejemplo, ciruela pasa sólo las Dense capas.
# Create a base model
base_model = setup_model()
base_model.load_weights(pretrained_weights) # optional but recommended for model accuracy
# Helper function uses `prune_low_magnitude` to make only the
# Dense layers train with pruning.
def apply_pruning_to_dense(layer):
if isinstance(layer, tf.keras.layers.Dense):
return tfmot.sparsity.keras.prune_low_magnitude(layer)
return layer
# Use `tf.keras.models.clone_model` to apply `apply_pruning_to_dense`
# to the layers of the model.
model_for_pruning = tf.keras.models.clone_model(
base_model,
clone_function=apply_pruning_to_dense,
)
model_for_pruning.summary()
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.bias WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details. Model: "sequential_3" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= prune_low_magnitude_dense_3 (None, 20) 822 _________________________________________________________________ flatten_3 (Flatten) (None, 20) 0 ================================================================= Total params: 822 Trainable params: 420 Non-trainable params: 402 _________________________________________________________________
Aunque este ejemplo utiliza el tipo de la capa de decidir qué podar, la forma más fácil de podar una capa determinada es establecer su name la propiedad, y el aspecto de ese nombre en la clone_function .
print(base_model.layers[0].name)
dense_3
Precisión de modelo más legible pero potencialmente más baja
Esto no es compatible con el ajuste fino con la poda, por lo que puede ser menos preciso que los ejemplos anteriores que admiten el ajuste fino.
Mientras prune_low_magnitude se puede aplicar mientras la definición del modelo inicial, la carga de los pesos después de que no funciona en el siguiente ejemplos.
Ejemplo funcional
# Use `prune_low_magnitude` to make the `Dense` layer train with pruning.
i = tf.keras.Input(shape=(20,))
x = tfmot.sparsity.keras.prune_low_magnitude(tf.keras.layers.Dense(10))(i)
o = tf.keras.layers.Flatten()(x)
model_for_pruning = tf.keras.Model(inputs=i, outputs=o)
model_for_pruning.summary()
Model: "functional_1" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= input_1 (InputLayer) [(None, 20)] 0 _________________________________________________________________ prune_low_magnitude_dense_4 (None, 10) 412 _________________________________________________________________ flatten_4 (Flatten) (None, 10) 0 ================================================================= Total params: 412 Trainable params: 210 Non-trainable params: 202 _________________________________________________________________
Ejemplo secuencial
# Use `prune_low_magnitude` to make the `Dense` layer train with pruning.
model_for_pruning = tf.keras.Sequential([
tfmot.sparsity.keras.prune_low_magnitude(tf.keras.layers.Dense(20, input_shape=input_shape)),
tf.keras.layers.Flatten()
])
model_for_pruning.summary()
Model: "sequential_4" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= prune_low_magnitude_dense_5 (None, 20) 822 _________________________________________________________________ flatten_5 (Flatten) (None, 20) 0 ================================================================= Total params: 822 Trainable params: 420 Non-trainable params: 402 _________________________________________________________________
Pode la capa personalizada de Keras o modifique partes de la capa para podar
Error común: la poda el sesgo por lo general perjudica la exactitud del modelo demasiado.
tfmot.sparsity.keras.PrunableLayer sirve para dos casos de uso:
- Podar una capa personalizada de Keras
- Modifique partes de una capa de Keras incorporada para podar.
Para un ejemplo, los valores predeterminados de API sólo para la poda el núcleo de la Dense capa. El siguiente ejemplo también elimina el sesgo.
class MyDenseLayer(tf.keras.layers.Dense, tfmot.sparsity.keras.PrunableLayer):
def get_prunable_weights(self):
# Prune bias also, though that usually harms model accuracy too much.
return [self.kernel, self.bias]
# Use `prune_low_magnitude` to make the `MyDenseLayer` layer train with pruning.
model_for_pruning = tf.keras.Sequential([
tfmot.sparsity.keras.prune_low_magnitude(MyDenseLayer(20, input_shape=input_shape)),
tf.keras.layers.Flatten()
])
model_for_pruning.summary()
Model: "sequential_5" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= prune_low_magnitude_my_dense (None, 20) 843 _________________________________________________________________ flatten_6 (Flatten) (None, 20) 0 ================================================================= Total params: 843 Trainable params: 420 Non-trainable params: 423 _________________________________________________________________
Modelo de tren
Model.fit
Llame a la tfmot.sparsity.keras.UpdatePruningStep devolución de llamada durante el entrenamiento.
Para la formación de depuración ayuda, utilice el tfmot.sparsity.keras.PruningSummaries de devolución de llamada.
# Define the model.
base_model = setup_model()
base_model.load_weights(pretrained_weights) # optional but recommended for model accuracy
model_for_pruning = tfmot.sparsity.keras.prune_low_magnitude(base_model)
log_dir = tempfile.mkdtemp()
callbacks = [
tfmot.sparsity.keras.UpdatePruningStep(),
# Log sparsity and other metrics in Tensorboard.
tfmot.sparsity.keras.PruningSummaries(log_dir=log_dir)
]
model_for_pruning.compile(
loss=tf.keras.losses.categorical_crossentropy,
optimizer='adam',
metrics=['accuracy']
)
model_for_pruning.fit(
x_train,
y_train,
callbacks=callbacks,
epochs=2,
)
#docs_infra: no_execute
%tensorboard --logdir={log_dir}
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.bias WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details. Epoch 1/2 1/1 [==============================] - 0s 3ms/step - loss: 1.2485 - accuracy: 0.0000e+00 Epoch 2/2 WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/ops/summary_ops_v2.py:1277: stop (from tensorflow.python.eager.profiler) is deprecated and will be removed after 2020-07-01. Instructions for updating: use `tf.profiler.experimental.stop` instead. 1/1 [==============================] - 0s 2ms/step - loss: 1.1999 - accuracy: 0.0000e+00
Para los usuarios que no son Colab, se puede ver los resultados de una ejecución anterior de este bloque de código en TensorBoard.dev .
Bucle de entrenamiento personalizado
Llame a la tfmot.sparsity.keras.UpdatePruningStep devolución de llamada durante el entrenamiento.
Para la formación de depuración ayuda, utilice el tfmot.sparsity.keras.PruningSummaries de devolución de llamada.
# Define the model.
base_model = setup_model()
base_model.load_weights(pretrained_weights) # optional but recommended for model accuracy
model_for_pruning = tfmot.sparsity.keras.prune_low_magnitude(base_model)
# Boilerplate
loss = tf.keras.losses.categorical_crossentropy
optimizer = tf.keras.optimizers.Adam()
log_dir = tempfile.mkdtemp()
unused_arg = -1
epochs = 2
batches = 1 # example is hardcoded so that the number of batches cannot change.
# Non-boilerplate.
model_for_pruning.optimizer = optimizer
step_callback = tfmot.sparsity.keras.UpdatePruningStep()
step_callback.set_model(model_for_pruning)
log_callback = tfmot.sparsity.keras.PruningSummaries(log_dir=log_dir) # Log sparsity and other metrics in Tensorboard.
log_callback.set_model(model_for_pruning)
step_callback.on_train_begin() # run pruning callback
for _ in range(epochs):
log_callback.on_epoch_begin(epoch=unused_arg) # run pruning callback
for _ in range(batches):
step_callback.on_train_batch_begin(batch=unused_arg) # run pruning callback
with tf.GradientTape() as tape:
logits = model_for_pruning(x_train, training=True)
loss_value = loss(y_train, logits)
grads = tape.gradient(loss_value, model_for_pruning.trainable_variables)
optimizer.apply_gradients(zip(grads, model_for_pruning.trainable_variables))
step_callback.on_epoch_end(batch=unused_arg) # run pruning callback
#docs_infra: no_execute
%tensorboard --logdir={log_dir}
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.bias WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.
Para los usuarios que no son Colab, se puede ver los resultados de una ejecución anterior de este bloque de código en TensorBoard.dev .
Mejorar la precisión del modelo podado
En primer lugar, vistazo a las tfmot.sparsity.keras.prune_low_magnitude documentación de la API para entender lo que es un programa de poda y los cálculos de cada tipo de poda horario.
consejos:
Tenga una tasa de aprendizaje que no sea demasiado alta ni demasiado baja cuando el modelo esté podando. Considere el horario de poda ser un hiperparámetro.
Como una prueba rápida, trata de experimentar con la poda de un modelo para la escasez final en el comienzo de la formación mediante el establecimiento de
begin_stepa 0 con untfmot.sparsity.keras.ConstantSparsityhorario. Puede tener suerte con buenos resultados.No pode con mucha frecuencia para que el modelo tenga tiempo de recuperarse. El calendario de poda proporciona una frecuencia predeterminada decente.
Para obtener ideas generales para mejorar la precisión del modelo, busque sugerencias para sus casos de uso en "Definir modelo".
Punto de control y deserialización
Debe conservar el paso del optimizador durante los puntos de control. Esto significa que si bien puede usar los modelos Keras HDF5 para puntos de control, no puede usar pesos Keras HDF5.
# Define the model.
base_model = setup_model()
base_model.load_weights(pretrained_weights) # optional but recommended for model accuracy
model_for_pruning = tfmot.sparsity.keras.prune_low_magnitude(base_model)
_, keras_model_file = tempfile.mkstemp('.h5')
# Checkpoint: saving the optimizer is necessary (include_optimizer=True is the default).
model_for_pruning.save(keras_model_file, include_optimizer=True)
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.bias WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.
Lo anterior se aplica generalmente. El código siguiente solo es necesario para el formato del modelo HDF5 (no los pesos HDF5 y otros formatos).
# Deserialize model.
with tfmot.sparsity.keras.prune_scope():
loaded_model = tf.keras.models.load_model(keras_model_file)
loaded_model.summary()
WARNING:tensorflow:No training configuration found in the save file, so the model was *not* compiled. Compile it manually. Model: "sequential_8" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= prune_low_magnitude_dense_8 (None, 20) 822 _________________________________________________________________ prune_low_magnitude_flatten_ (None, 20) 1 ================================================================= Total params: 823 Trainable params: 420 Non-trainable params: 403 _________________________________________________________________
Implementar modelo podado
Exportar modelo con compresión de tamaño
Error común: ambos strip_pruning y la aplicación de un algoritmo de compresión estándar (por ejemplo, a través de gzip) son necesarias para ver las ventajas de la compresión de la poda.
# Define the model.
base_model = setup_model()
base_model.load_weights(pretrained_weights) # optional but recommended for model accuracy
model_for_pruning = tfmot.sparsity.keras.prune_low_magnitude(base_model)
# Typically you train the model here.
model_for_export = tfmot.sparsity.keras.strip_pruning(model_for_pruning)
print("final model")
model_for_export.summary()
print("\n")
print("Size of gzipped pruned model without stripping: %.2f bytes" % (get_gzipped_model_size(model_for_pruning)))
print("Size of gzipped pruned model with stripping: %.2f bytes" % (get_gzipped_model_size(model_for_export)))
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.bias WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details. final model Model: "sequential_9" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= dense_9 (Dense) (None, 20) 420 _________________________________________________________________ flatten_10 (Flatten) (None, 20) 0 ================================================================= Total params: 420 Trainable params: 420 Non-trainable params: 0 _________________________________________________________________ Size of gzipped pruned model without stripping: 3299.00 bytes Size of gzipped pruned model with stripping: 2876.00 bytes
Optimizaciones específicas de hardware
Una vez diferentes backends permiten la poda para mejorar la latencia , utilizando el bloque escasez puede mejorar la latencia para un determinado hardware.
Aumentar el tamaño del bloque disminuirá la dispersión máxima que se puede lograr para la precisión del modelo de destino. A pesar de esto, la latencia aún puede mejorar.
Para más detalles sobre lo que está apoyado por el bloque escasez, consulte las tfmot.sparsity.keras.prune_low_magnitude documentación de la API.
base_model = setup_model()
# For using intrinsics on a CPU with 128-bit registers, together with 8-bit
# quantized weights, a 1x16 block size is nice because the block perfectly
# fits into the register.
pruning_params = {'block_size': [1, 16]}
model_for_pruning = tfmot.sparsity.keras.prune_low_magnitude(base_model, **pruning_params)
model_for_pruning.summary()
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2 WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'm' for (root).layer_with_weights-0.bias WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.kernel WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer's state 'v' for (root).layer_with_weights-0.bias WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details. Model: "sequential_10" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= prune_low_magnitude_dense_10 (None, 20) 822 _________________________________________________________________ prune_low_magnitude_flatten_ (None, 20) 1 ================================================================= Total params: 823 Trainable params: 420 Non-trainable params: 403 _________________________________________________________________