 在 TensorFlow.org 上查看 |
 在 Google Colab 中运行 |
 在 GitHub 上查看源代码 |
 下载笔记本 |
概述
欢迎阅读 TensorFlow Model Optimization Toolkit 中权重聚类的端到端示例。
其他页面
有关权重聚类的定义以及如何确定是否应使用权重聚类（包括支持的功能）的介绍，请参阅概述页面。
要快速找到您的用例（不局限于使用 16 个簇完全聚类模型）所需的 API，请参阅综合指南。
目录
在本教程中，您将：
- 从头开始为 MNIST 数据集训练一个
tf.keras模型。 - 通过应用权重聚类 API 对模型进行微调，并查看准确率。
- 通过聚类创建一个大小缩减至六分之一的 TF 和 TFLite 模型。
- 通过将权重聚类与训练后量化相结合，创建一个大小缩减至八分之一的 TFLite 模型。
- 查看从 TF 到 TFLite 的准确率持久性。
设置
您可以在本地 virtualenv 或 Colab 中运行此 Jupyter 笔记本。有关设置依赖项的详细信息，请参阅安装指南。
pip install -q tensorflow-model-optimizationimport tensorflow as tf
from tensorflow import keras
import numpy as np
import tempfile
import zipfile
import os
在不使用聚类的情况下为 MNIST 训练 tf.keras 模型
# Load MNIST dataset
mnist = keras.datasets.mnist
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()
# Normalize the input image so that each pixel value is between 0 to 1.
train_images = train_images / 255.0
test_images = test_images / 255.0
# Define the model architecture.
model = keras.Sequential([
keras.layers.InputLayer(input_shape=(28, 28)),
keras.layers.Reshape(target_shape=(28, 28, 1)),
keras.layers.Conv2D(filters=12, kernel_size=(3, 3), activation=tf.nn.relu),
keras.layers.MaxPooling2D(pool_size=(2, 2)),
keras.layers.Flatten(),
keras.layers.Dense(10)
])
# Train the digit classification model
model.compile(optimizer='adam',
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
model.fit(
train_images,
train_labels,
validation_split=0.1,
epochs=10
)
Epoch 1/10 1/1688 [..............................] - ETA: 1s - loss: 2.3329 - accuracy: 0.0938 24/1688 [..............................] - ETA: 3s - loss: 2.0253 - accuracy: 0.4297 48/1688 [..............................] - ETA: 3s - loss: 1.6723 - accuracy: 0.6055 72/1688 [>.............................] - ETA: 3s - loss: 1.3881 - accuracy: 0.6719 96/1688 [>.............................] - ETA: 3s - loss: 1.1854 - accuracy: 0.7148 120/1688 [=>............................] - ETA: 3s - loss: 1.0494 - accuracy: 0.7445 145/1688 [=>............................] - ETA: 3s - loss: 0.9444 - accuracy: 0.7677 170/1688 [==>...........................] - ETA: 3s - loss: 0.8710 - accuracy: 0.7824 194/1688 [==>...........................] - ETA: 3s - loss: 0.8102 - accuracy: 0.7956 218/1688 [==>...........................] - ETA: 3s - loss: 0.7698 - accuracy: 0.8045 242/1688 [===>..........................] - ETA: 3s - loss: 0.7314 - accuracy: 0.8129 266/1688 [===>..........................] - ETA: 2s - loss: 0.6957 - accuracy: 0.8217 292/1688 [====>.........................] - ETA: 2s - loss: 0.6652 - accuracy: 0.8283 318/1688 [====>.........................] - ETA: 2s - loss: 0.6395 - accuracy: 0.8333 344/1688 [=====>........................] - ETA: 2s - loss: 0.6116 - accuracy: 0.8396 370/1688 [=====>........................] - ETA: 2s - loss: 0.5926 - accuracy: 0.8440 397/1688 [======>.......................] - ETA: 2s - loss: 0.5738 - accuracy: 0.8488 424/1688 [======>.......................] - ETA: 2s - loss: 0.5559 - accuracy: 0.8522 451/1688 [=======>......................] - ETA: 2s - loss: 0.5439 - accuracy: 0.8548 478/1688 [=======>......................] - ETA: 2s - loss: 0.5294 - accuracy: 0.8585 505/1688 [=======>......................] - ETA: 2s - loss: 0.5170 - accuracy: 0.8610 532/1688 [========>.....................] - ETA: 2s - loss: 0.5033 - accuracy: 0.8644 559/1688 [========>.....................] - ETA: 2s - loss: 0.4915 - accuracy: 0.8672 586/1688 [=========>....................] - ETA: 2s - loss: 0.4806 - accuracy: 0.8699 613/1688 [=========>....................] - ETA: 2s - loss: 0.4727 - accuracy: 0.8720 640/1688 [==========>...................] - ETA: 2s - loss: 0.4652 - accuracy: 0.8738 667/1688 [==========>...................] - ETA: 2s - loss: 0.4573 - accuracy: 0.8761 694/1688 [===========>..................] - ETA: 1s - loss: 0.4496 - accuracy: 0.8781 721/1688 [===========>..................] - ETA: 1s - loss: 0.4421 - accuracy: 0.8798 748/1688 [============>.................] - ETA: 1s - loss: 0.4352 - accuracy: 0.8817 775/1688 [============>.................] - ETA: 1s - loss: 0.4272 - accuracy: 0.8835 802/1688 [=============>................] - ETA: 1s - loss: 0.4208 - accuracy: 0.8852 828/1688 [=============>................] - ETA: 1s - loss: 0.4149 - accuracy: 0.8867 855/1688 [==============>...............] - ETA: 1s - loss: 0.4079 - accuracy: 0.8883 881/1688 [==============>...............] - ETA: 1s - loss: 0.4016 - accuracy: 0.8901 907/1688 [===============>..............] - ETA: 1s - loss: 0.3960 - accuracy: 0.8916 934/1688 [===============>..............] - ETA: 1s - loss: 0.3904 - accuracy: 0.8929 961/1688 [================>.............] - ETA: 1s - loss: 0.3857 - accuracy: 0.8940 988/1688 [================>.............] - ETA: 1s - loss: 0.3814 - accuracy: 0.8951 1015/1688 [=================>............] - ETA: 1s - loss: 0.3776 - accuracy: 0.8963 1041/1688 [=================>............] - ETA: 1s - loss: 0.3729 - accuracy: 0.8976 1068/1688 [=================>............] - ETA: 1s - loss: 0.3690 - accuracy: 0.8986 1095/1688 [==================>...........] - ETA: 1s - loss: 0.3649 - accuracy: 0.8997 1122/1688 [==================>...........] - ETA: 1s - loss: 0.3615 - accuracy: 0.9007 1149/1688 [===================>..........] - ETA: 1s - loss: 0.3589 - accuracy: 0.9011 1176/1688 [===================>..........] - ETA: 1s - loss: 0.3550 - accuracy: 0.9021 1202/1688 [====================>.........] - ETA: 0s - loss: 0.3517 - accuracy: 0.9028 1229/1688 [====================>.........] - ETA: 0s - loss: 0.3486 - accuracy: 0.9034 1256/1688 [=====================>........] - ETA: 0s - loss: 0.3438 - accuracy: 0.9048 1283/1688 [=====================>........] - ETA: 0s - loss: 0.3406 - accuracy: 0.9057 1310/1688 [======================>.......] - ETA: 0s - loss: 0.3379 - accuracy: 0.9065 1337/1688 [======================>.......] - ETA: 0s - loss: 0.3340 - accuracy: 0.9075 1363/1688 [=======================>......] - ETA: 0s - loss: 0.3312 - accuracy: 0.9082 1390/1688 [=======================>......] - ETA: 0s - loss: 0.3284 - accuracy: 0.9090 1417/1688 [========================>.....] - ETA: 0s - loss: 0.3258 - accuracy: 0.9098 1444/1688 [========================>.....] - ETA: 0s - loss: 0.3232 - accuracy: 0.9104 1470/1688 [=========================>....] - ETA: 0s - loss: 0.3204 - accuracy: 0.9111 1496/1688 [=========================>....] - ETA: 0s - loss: 0.3172 - accuracy: 0.9121 1522/1688 [==========================>...] - ETA: 0s - loss: 0.3143 - accuracy: 0.9128 1549/1688 [==========================>...] - ETA: 0s - loss: 0.3118 - accuracy: 0.9134 1576/1688 [===========================>..] - ETA: 0s - loss: 0.3090 - accuracy: 0.9140 1603/1688 [===========================>..] - ETA: 0s - loss: 0.3063 - accuracy: 0.9149 1629/1688 [===========================>..] - ETA: 0s - loss: 0.3046 - accuracy: 0.9154 1656/1688 [============================>.] - ETA: 0s - loss: 0.3028 - accuracy: 0.9160 1683/1688 [============================>.] - ETA: 0s - loss: 0.3007 - accuracy: 0.9166 1688/1688 [==============================] - 4s 2ms/step - loss: 0.3002 - accuracy: 0.9167 - val_loss: 0.1301 - val_accuracy: 0.9625 Epoch 2/10 1/1688 [..............................] - ETA: 0s - loss: 0.1073 - accuracy: 0.9375 28/1688 [..............................] - ETA: 3s - loss: 0.1506 - accuracy: 0.9464 55/1688 [..............................] - ETA: 3s - loss: 0.1486 - accuracy: 0.9500 82/1688 [>.............................] - ETA: 3s - loss: 0.1503 - accuracy: 0.9554 109/1688 [>.............................] - ETA: 2s - loss: 0.1558 - accuracy: 0.9538 136/1688 [=>............................] - ETA: 2s - loss: 0.1547 - accuracy: 0.9554 163/1688 [=>............................] - ETA: 2s - loss: 0.1486 - accuracy: 0.9578 190/1688 [==>...........................] - ETA: 2s - loss: 0.1457 - accuracy: 0.9595 217/1688 [==>...........................] - ETA: 2s - loss: 0.1468 - accuracy: 0.9590 244/1688 [===>..........................] - ETA: 2s - loss: 0.1431 - accuracy: 0.9598 271/1688 [===>..........................] - ETA: 2s - loss: 0.1472 - accuracy: 0.9576 297/1688 [====>.........................] - ETA: 2s - loss: 0.1452 - accuracy: 0.9583 324/1688 [====>.........................] - ETA: 2s - loss: 0.1464 - accuracy: 0.9580 350/1688 [=====>........................] - ETA: 2s - loss: 0.1462 - accuracy: 0.9581 377/1688 [=====>........................] - ETA: 2s - loss: 0.1465 - accuracy: 0.9581 404/1688 [======>.......................] - ETA: 2s - loss: 0.1471 - accuracy: 0.9580 430/1688 [======>.......................] - ETA: 2s - loss: 0.1493 - accuracy: 0.9571 457/1688 [=======>......................] - ETA: 2s - loss: 0.1506 - accuracy: 0.9568 484/1688 [=======>......................] - ETA: 2s - loss: 0.1491 - accuracy: 0.9572 511/1688 [========>.....................] - ETA: 2s - loss: 0.1493 - accuracy: 0.9571 538/1688 [========>.....................] - ETA: 2s - loss: 0.1499 - accuracy: 0.9572 564/1688 [=========>....................] - ETA: 2s - loss: 0.1491 - accuracy: 0.9571 591/1688 [=========>....................] - ETA: 2s - loss: 0.1485 - accuracy: 0.9576 618/1688 [=========>....................] - ETA: 2s - loss: 0.1474 - accuracy: 0.9579 645/1688 [==========>...................] - ETA: 1s - loss: 0.1472 - accuracy: 0.9575 672/1688 [==========>...................] - ETA: 1s - loss: 0.1456 - accuracy: 0.9581 698/1688 [===========>..................] - ETA: 1s - loss: 0.1447 - accuracy: 0.9585 724/1688 [===========>..................] - ETA: 1s - loss: 0.1427 - accuracy: 0.9590 750/1688 [============>.................] - ETA: 1s - loss: 0.1414 - accuracy: 0.9594 776/1688 [============>.................] - ETA: 1s - loss: 0.1404 - accuracy: 0.9596 803/1688 [=============>................] - ETA: 1s - loss: 0.1396 - accuracy: 0.9599 830/1688 [=============>................] - ETA: 1s - loss: 0.1383 - accuracy: 0.9602 856/1688 [==============>...............] - ETA: 1s - loss: 0.1372 - accuracy: 0.9604 883/1688 [==============>...............] - ETA: 1s - loss: 0.1375 - accuracy: 0.9601 909/1688 [===============>..............] - ETA: 1s - loss: 0.1371 - accuracy: 0.9603 935/1688 [===============>..............] - ETA: 1s - loss: 0.1363 - accuracy: 0.9605 961/1688 [================>.............] - ETA: 1s - loss: 0.1356 - accuracy: 0.9607 987/1688 [================>.............] - ETA: 1s - loss: 0.1352 - accuracy: 0.9609 1014/1688 [=================>............] - ETA: 1s - loss: 0.1346 - accuracy: 0.9610 1041/1688 [=================>............] - ETA: 1s - loss: 0.1344 - accuracy: 0.9608 1067/1688 [=================>............] - ETA: 1s - loss: 0.1336 - accuracy: 0.9611 1093/1688 [==================>...........] - ETA: 1s - loss: 0.1341 - accuracy: 0.9611 1120/1688 [==================>...........] - ETA: 1s - loss: 0.1337 - accuracy: 0.9613 1147/1688 [===================>..........] - ETA: 1s - loss: 0.1333 - accuracy: 0.9614 1173/1688 [===================>..........] - ETA: 0s - loss: 0.1326 - accuracy: 0.9617 1200/1688 [====================>.........] - ETA: 0s - loss: 0.1320 - accuracy: 0.9617 1226/1688 [====================>.........] - ETA: 0s - loss: 0.1320 - accuracy: 0.9618 1253/1688 [=====================>........] - ETA: 0s - loss: 0.1315 - accuracy: 0.9621 1279/1688 [=====================>........] - ETA: 0s - loss: 0.1309 - accuracy: 0.9623 1305/1688 [======================>.......] - ETA: 0s - loss: 0.1300 - accuracy: 0.9625 1331/1688 [======================>.......] - ETA: 0s - loss: 0.1291 - accuracy: 0.9627 1357/1688 [=======================>......] - ETA: 0s - loss: 0.1290 - accuracy: 0.9626 1384/1688 [=======================>......] - ETA: 0s - loss: 0.1298 - accuracy: 0.9622 1411/1688 [========================>.....] - ETA: 0s - loss: 0.1290 - accuracy: 0.9625 1437/1688 [========================>.....] - ETA: 0s - loss: 0.1287 - accuracy: 0.9625 1463/1688 [=========================>....] - ETA: 0s - loss: 0.1281 - accuracy: 0.9628 1489/1688 [=========================>....] - ETA: 0s - loss: 0.1277 - accuracy: 0.9629 1515/1688 [=========================>....] - ETA: 0s - loss: 0.1270 - accuracy: 0.9631 1542/1688 [==========================>...] - ETA: 0s - loss: 0.1267 - accuracy: 0.9632 1568/1688 [==========================>...] - ETA: 0s - loss: 0.1267 - accuracy: 0.9633 1595/1688 [===========================>..] - ETA: 0s - loss: 0.1264 - accuracy: 0.9635 1622/1688 [===========================>..] - ETA: 0s - loss: 0.1256 - accuracy: 0.9638 1649/1688 [============================>.] - ETA: 0s - loss: 0.1251 - accuracy: 0.9639 1676/1688 [============================>.] - ETA: 0s - loss: 0.1253 - accuracy: 0.9639 1688/1688 [==============================] - 4s 2ms/step - loss: 0.1251 - accuracy: 0.9639 - val_loss: 0.0870 - val_accuracy: 0.9773 Epoch 3/10 1/1688 [..............................] - ETA: 0s - loss: 0.0485 - accuracy: 1.0000 27/1688 [..............................] - ETA: 3s - loss: 0.0927 - accuracy: 0.9745 53/1688 [..............................] - ETA: 3s - loss: 0.1039 - accuracy: 0.9717 80/1688 [>.............................] - ETA: 3s - loss: 0.1035 - accuracy: 0.9730 107/1688 [>.............................] - ETA: 3s - loss: 0.0950 - accuracy: 0.9749 134/1688 [=>............................] - ETA: 2s - loss: 0.0960 - accuracy: 0.9739 160/1688 [=>............................] - ETA: 2s - loss: 0.0981 - accuracy: 0.9732 187/1688 [==>...........................] - ETA: 2s - loss: 0.0968 - accuracy: 0.9729 213/1688 [==>...........................] - ETA: 2s - loss: 0.0953 - accuracy: 0.9732 240/1688 [===>..........................] - ETA: 2s - loss: 0.0947 - accuracy: 0.9729 266/1688 [===>..........................] - ETA: 2s - loss: 0.0952 - accuracy: 0.9725 293/1688 [====>.........................] - ETA: 2s - loss: 0.0929 - accuracy: 0.9734 319/1688 [====>.........................] - ETA: 2s - loss: 0.0917 - accuracy: 0.9737 346/1688 [=====>........................] - ETA: 2s - loss: 0.0900 - accuracy: 0.9745 373/1688 [=====>........................] - ETA: 2s - loss: 0.0889 - accuracy: 0.9751 399/1688 [======>.......................] - ETA: 2s - loss: 0.0880 - accuracy: 0.9749 426/1688 [======>.......................] - ETA: 2s - loss: 0.0889 - accuracy: 0.9752 453/1688 [=======>......................] - ETA: 2s - loss: 0.0891 - accuracy: 0.9748 480/1688 [=======>......................] - ETA: 2s - loss: 0.0890 - accuracy: 0.9749 507/1688 [========>.....................] - ETA: 2s - loss: 0.0890 - accuracy: 0.9747 534/1688 [========>.....................] - ETA: 2s - loss: 0.0888 - accuracy: 0.9746 560/1688 [========>.....................] - ETA: 2s - loss: 0.0892 - accuracy: 0.9745 587/1688 [=========>....................] - ETA: 2s - loss: 0.0897 - accuracy: 0.9744 614/1688 [=========>....................] - ETA: 2s - loss: 0.0894 - accuracy: 0.9747 641/1688 [==========>...................] - ETA: 2s - loss: 0.0893 - accuracy: 0.9747 668/1688 [==========>...................] - ETA: 1s - loss: 0.0882 - accuracy: 0.9752 695/1688 [===========>..................] - ETA: 1s - loss: 0.0882 - accuracy: 0.9748 722/1688 [===========>..................] - ETA: 1s - loss: 0.0883 - accuracy: 0.9745 749/1688 [============>.................] - ETA: 1s - loss: 0.0883 - accuracy: 0.9745 776/1688 [============>.................] - ETA: 1s - loss: 0.0886 - accuracy: 0.9744 803/1688 [=============>................] - ETA: 1s - loss: 0.0889 - accuracy: 0.9742 830/1688 [=============>................] - ETA: 1s - loss: 0.0883 - accuracy: 0.9744 857/1688 [==============>...............] - ETA: 1s - loss: 0.0888 - accuracy: 0.9743 884/1688 [==============>...............] - ETA: 1s - loss: 0.0883 - accuracy: 0.9744 911/1688 [===============>..............] - ETA: 1s - loss: 0.0879 - accuracy: 0.9744 938/1688 [===============>..............] - ETA: 1s - loss: 0.0883 - accuracy: 0.9744 965/1688 [================>.............] - ETA: 1s - loss: 0.0878 - accuracy: 0.9744 992/1688 [================>.............] - ETA: 1s - loss: 0.0877 - accuracy: 0.9745 1018/1688 [=================>............] - ETA: 1s - loss: 0.0882 - accuracy: 0.9745 1044/1688 [=================>............] - ETA: 1s - loss: 0.0878 - accuracy: 0.9745 1070/1688 [==================>...........] - ETA: 1s - loss: 0.0885 - accuracy: 0.9744 1096/1688 [==================>...........] - ETA: 1s - loss: 0.0883 - accuracy: 0.9744 1122/1688 [==================>...........] - ETA: 1s - loss: 0.0880 - accuracy: 0.9745 1148/1688 [===================>..........] - ETA: 1s - loss: 0.0874 - accuracy: 0.9747 1174/1688 [===================>..........] - ETA: 0s - loss: 0.0879 - accuracy: 0.9745 1200/1688 [====================>.........] - ETA: 0s - loss: 0.0880 - accuracy: 0.9747 1227/1688 [====================>.........] - ETA: 0s - loss: 0.0879 - accuracy: 0.9746 1253/1688 [=====================>........] - ETA: 0s - loss: 0.0879 - accuracy: 0.9746 1279/1688 [=====================>........] - ETA: 0s - loss: 0.0886 - accuracy: 0.9745 1306/1688 [======================>.......] - ETA: 0s - loss: 0.0894 - accuracy: 0.9743 1333/1688 [======================>.......] - ETA: 0s - loss: 0.0896 - accuracy: 0.9742 1359/1688 [=======================>......] - ETA: 0s - loss: 0.0894 - accuracy: 0.9742 1385/1688 [=======================>......] - ETA: 0s - loss: 0.0891 - accuracy: 0.9744 1412/1688 [========================>.....] - ETA: 0s - loss: 0.0890 - accuracy: 0.9744 1439/1688 [========================>.....] - ETA: 0s - loss: 0.0886 - accuracy: 0.9744 1466/1688 [=========================>....] - ETA: 0s - loss: 0.0892 - accuracy: 0.9743 1493/1688 [=========================>....] - ETA: 0s - loss: 0.0893 - accuracy: 0.9743 1519/1688 [=========================>....] - ETA: 0s - loss: 0.0892 - accuracy: 0.9742 1546/1688 [==========================>...] - ETA: 0s - loss: 0.0891 - accuracy: 0.9742 1572/1688 [==========================>...] - ETA: 0s - loss: 0.0885 - accuracy: 0.9743 1598/1688 [===========================>..] - ETA: 0s - loss: 0.0882 - accuracy: 0.9745 1624/1688 [===========================>..] - ETA: 0s - loss: 0.0881 - accuracy: 0.9744 1651/1688 [============================>.] - ETA: 0s - loss: 0.0882 - accuracy: 0.9743 1678/1688 [============================>.] - ETA: 0s - loss: 0.0887 - accuracy: 0.9741 1688/1688 [==============================] - 4s 2ms/step - loss: 0.0890 - accuracy: 0.9740 - val_loss: 0.0697 - val_accuracy: 0.9812 Epoch 4/10 1/1688 [..............................] - ETA: 0s - loss: 0.2472 - accuracy: 0.9375 28/1688 [..............................] - ETA: 3s - loss: 0.0724 - accuracy: 0.9777 55/1688 [..............................] - ETA: 3s - loss: 0.0712 - accuracy: 0.9790 82/1688 [>.............................] - ETA: 3s - loss: 0.0671 - accuracy: 0.9832 108/1688 [>.............................] - ETA: 2s - loss: 0.0618 - accuracy: 0.9844 134/1688 [=>............................] - ETA: 2s - loss: 0.0585 - accuracy: 0.9851 160/1688 [=>............................] - ETA: 2s - loss: 0.0598 - accuracy: 0.9855 186/1688 [==>...........................] - ETA: 2s - loss: 0.0614 - accuracy: 0.9845 212/1688 [==>...........................] - ETA: 2s - loss: 0.0634 - accuracy: 0.9844 238/1688 [===>..........................] - ETA: 2s - loss: 0.0649 - accuracy: 0.9835 264/1688 [===>..........................] - ETA: 2s - loss: 0.0639 - accuracy: 0.9839 291/1688 [====>.........................] - ETA: 2s - loss: 0.0634 - accuracy: 0.9836 317/1688 [====>.........................] - ETA: 2s - loss: 0.0647 - accuracy: 0.9828 343/1688 [=====>........................] - ETA: 2s - loss: 0.0646 - accuracy: 0.9825 369/1688 [=====>........................] - ETA: 2s - loss: 0.0677 - accuracy: 0.9815 396/1688 [======>.......................] - ETA: 2s - loss: 0.0675 - accuracy: 0.9815 423/1688 [======>.......................] - ETA: 2s - loss: 0.0667 - accuracy: 0.9816 450/1688 [======>.......................] - ETA: 2s - loss: 0.0660 - accuracy: 0.9816 477/1688 [=======>......................] - ETA: 2s - loss: 0.0670 - accuracy: 0.9812 503/1688 [=======>......................] - ETA: 2s - loss: 0.0674 - accuracy: 0.9809 530/1688 [========>.....................] - ETA: 2s - loss: 0.0687 - accuracy: 0.9806 557/1688 [========>.....................] - ETA: 2s - loss: 0.0690 - accuracy: 0.9805 584/1688 [=========>....................] - ETA: 2s - loss: 0.0690 - accuracy: 0.9805 611/1688 [=========>....................] - ETA: 2s - loss: 0.0691 - accuracy: 0.9806 638/1688 [==========>...................] - ETA: 2s - loss: 0.0682 - accuracy: 0.9808 665/1688 [==========>...................] - ETA: 1s - loss: 0.0677 - accuracy: 0.9808 692/1688 [===========>..................] - ETA: 1s - loss: 0.0678 - accuracy: 0.9806 719/1688 [===========>..................] - ETA: 1s - loss: 0.0679 - accuracy: 0.9807 746/1688 [============>.................] - ETA: 1s - loss: 0.0679 - accuracy: 0.9805 772/1688 [============>.................] - ETA: 1s - loss: 0.0682 - accuracy: 0.9807 799/1688 [=============>................] - ETA: 1s - loss: 0.0680 - accuracy: 0.9806 825/1688 [=============>................] - ETA: 1s - loss: 0.0681 - accuracy: 0.9805 852/1688 [==============>...............] - ETA: 1s - loss: 0.0686 - accuracy: 0.9804 879/1688 [==============>...............] - ETA: 1s - loss: 0.0691 - accuracy: 0.9803 905/1688 [===============>..............] - ETA: 1s - loss: 0.0693 - accuracy: 0.9801 932/1688 [===============>..............] - ETA: 1s - loss: 0.0699 - accuracy: 0.9798 959/1688 [================>.............] - ETA: 1s - loss: 0.0704 - accuracy: 0.9797 985/1688 [================>.............] - ETA: 1s - loss: 0.0705 - accuracy: 0.9796 1011/1688 [================>.............] - ETA: 1s - loss: 0.0703 - accuracy: 0.9797 1037/1688 [=================>............] - ETA: 1s - loss: 0.0703 - accuracy: 0.9797 1064/1688 [=================>............] - ETA: 1s - loss: 0.0707 - accuracy: 0.9796 1091/1688 [==================>...........] - ETA: 1s - loss: 0.0708 - accuracy: 0.9794 1118/1688 [==================>...........] - ETA: 1s - loss: 0.0706 - accuracy: 0.9795 1145/1688 [===================>..........] - ETA: 1s - loss: 0.0710 - accuracy: 0.9793 1172/1688 [===================>..........] - ETA: 0s - loss: 0.0708 - accuracy: 0.9793 1199/1688 [====================>.........] - ETA: 0s - loss: 0.0704 - accuracy: 0.9794 1225/1688 [====================>.........] - ETA: 0s - loss: 0.0709 - accuracy: 0.9791 1252/1688 [=====================>........] - ETA: 0s - loss: 0.0715 - accuracy: 0.9791 1278/1688 [=====================>........] - ETA: 0s - loss: 0.0716 - accuracy: 0.9789 1305/1688 [======================>.......] - ETA: 0s - loss: 0.0715 - accuracy: 0.9788 1331/1688 [======================>.......] - ETA: 0s - loss: 0.0715 - accuracy: 0.9789 1358/1688 [=======================>......] - ETA: 0s - loss: 0.0721 - accuracy: 0.9787 1385/1688 [=======================>......] - ETA: 0s - loss: 0.0721 - accuracy: 0.9787 1412/1688 [========================>.....] - ETA: 0s - loss: 0.0722 - accuracy: 0.9786 1439/1688 [========================>.....] - ETA: 0s - loss: 0.0724 - accuracy: 0.9784 1466/1688 [=========================>....] - ETA: 0s - loss: 0.0729 - accuracy: 0.9784 1493/1688 [=========================>....] - ETA: 0s - loss: 0.0732 - accuracy: 0.9783 1520/1688 [==========================>...] - ETA: 0s - loss: 0.0735 - accuracy: 0.9782 1547/1688 [==========================>...] - ETA: 0s - loss: 0.0735 - accuracy: 0.9782 1574/1688 [==========================>...] - ETA: 0s - loss: 0.0731 - accuracy: 0.9783 1601/1688 [===========================>..] - ETA: 0s - loss: 0.0730 - accuracy: 0.9784 1628/1688 [===========================>..] - ETA: 0s - loss: 0.0730 - accuracy: 0.9784 1655/1688 [============================>.] - ETA: 0s - loss: 0.0731 - accuracy: 0.9783 1682/1688 [============================>.] - ETA: 0s - loss: 0.0726 - accuracy: 0.9785 1688/1688 [==============================] - 4s 2ms/step - loss: 0.0725 - accuracy: 0.9786 - val_loss: 0.0643 - val_accuracy: 0.9828 Epoch 5/10 1/1688 [..............................] - ETA: 0s - loss: 0.0057 - accuracy: 1.0000 28/1688 [..............................] - ETA: 3s - loss: 0.0615 - accuracy: 0.9810 54/1688 [..............................] - ETA: 3s - loss: 0.0647 - accuracy: 0.9826 80/1688 [>.............................] - ETA: 3s - loss: 0.0615 - accuracy: 0.9832 106/1688 [>.............................] - ETA: 3s - loss: 0.0621 - accuracy: 0.9832 132/1688 [=>............................] - ETA: 3s - loss: 0.0593 - accuracy: 0.9832 159/1688 [=>............................] - ETA: 2s - loss: 0.0575 - accuracy: 0.9829 186/1688 [==>...........................] - ETA: 2s - loss: 0.0594 - accuracy: 0.9820 213/1688 [==>...........................] - ETA: 2s - loss: 0.0625 - accuracy: 0.9811 240/1688 [===>..........................] - ETA: 2s - loss: 0.0605 - accuracy: 0.9818 266/1688 [===>..........................] - ETA: 2s - loss: 0.0609 - accuracy: 0.9813 293/1688 [====>.........................] - ETA: 2s - loss: 0.0634 - accuracy: 0.9810 320/1688 [====>.........................] - ETA: 2s - loss: 0.0631 - accuracy: 0.9811 347/1688 [=====>........................] - ETA: 2s - loss: 0.0616 - accuracy: 0.9812 374/1688 [=====>........................] - ETA: 2s - loss: 0.0611 - accuracy: 0.9809 401/1688 [======>.......................] - ETA: 2s - loss: 0.0622 - accuracy: 0.9807 428/1688 [======>.......................] - ETA: 2s - loss: 0.0613 - accuracy: 0.9809 455/1688 [=======>......................] - ETA: 2s - loss: 0.0613 - accuracy: 0.9810 482/1688 [=======>......................] - ETA: 2s - loss: 0.0621 - accuracy: 0.9808 509/1688 [========>.....................] - ETA: 2s - loss: 0.0616 - accuracy: 0.9808 536/1688 [========>.....................] - ETA: 2s - loss: 0.0619 - accuracy: 0.9808 563/1688 [=========>....................] - ETA: 2s - loss: 0.0617 - accuracy: 0.9809 590/1688 [=========>....................] - ETA: 2s - loss: 0.0615 - accuracy: 0.9808 617/1688 [=========>....................] - ETA: 2s - loss: 0.0623 - accuracy: 0.9807 643/1688 [==========>...................] - ETA: 1s - loss: 0.0627 - accuracy: 0.9804 670/1688 [==========>...................] - ETA: 1s - loss: 0.0623 - accuracy: 0.9804 697/1688 [===========>..................] - ETA: 1s - loss: 0.0613 - accuracy: 0.9806 723/1688 [===========>..................] - ETA: 1s - loss: 0.0614 - accuracy: 0.9806 749/1688 [============>.................] - ETA: 1s - loss: 0.0619 - accuracy: 0.9805 775/1688 [============>.................] - ETA: 1s - loss: 0.0623 - accuracy: 0.9805 802/1688 [=============>................] - ETA: 1s - loss: 0.0622 - accuracy: 0.9806 829/1688 [=============>................] - ETA: 1s - loss: 0.0620 - accuracy: 0.9807 856/1688 [==============>...............] - ETA: 1s - loss: 0.0618 - accuracy: 0.9808 883/1688 [==============>...............] - ETA: 1s - loss: 0.0615 - accuracy: 0.9809 910/1688 [===============>..............] - ETA: 1s - loss: 0.0612 - accuracy: 0.9810 937/1688 [===============>..............] - ETA: 1s - loss: 0.0609 - accuracy: 0.9812 964/1688 [================>.............] - ETA: 1s - loss: 0.0602 - accuracy: 0.9814 991/1688 [================>.............] - ETA: 1s - loss: 0.0603 - accuracy: 0.9812 1018/1688 [=================>............] - ETA: 1s - loss: 0.0611 - accuracy: 0.9810 1045/1688 [=================>............] - ETA: 1s - loss: 0.0616 - accuracy: 0.9809 1072/1688 [==================>...........] - ETA: 1s - loss: 0.0615 - accuracy: 0.9809 1099/1688 [==================>...........] - ETA: 1s - loss: 0.0619 - accuracy: 0.9807 1126/1688 [===================>..........] - ETA: 1s - loss: 0.0619 - accuracy: 0.9806 1153/1688 [===================>..........] - ETA: 1s - loss: 0.0616 - accuracy: 0.9807 1180/1688 [===================>..........] - ETA: 0s - loss: 0.0616 - accuracy: 0.9808 1207/1688 [====================>.........] - ETA: 0s - loss: 0.0614 - accuracy: 0.9809 1234/1688 [====================>.........] - ETA: 0s - loss: 0.0611 - accuracy: 0.9810 1261/1688 [=====================>........] - ETA: 0s - loss: 0.0606 - accuracy: 0.9812 1288/1688 [=====================>........] - ETA: 0s - loss: 0.0605 - accuracy: 0.9812 1315/1688 [======================>.......] - ETA: 0s - loss: 0.0602 - accuracy: 0.9812 1341/1688 [======================>.......] - ETA: 0s - loss: 0.0601 - accuracy: 0.9813 1369/1688 [=======================>......] - ETA: 0s - loss: 0.0603 - accuracy: 0.9812 1396/1688 [=======================>......] - ETA: 0s - loss: 0.0602 - accuracy: 0.9812 1423/1688 [========================>.....] - ETA: 0s - loss: 0.0602 - accuracy: 0.9813 1450/1688 [========================>.....] - ETA: 0s - loss: 0.0609 - accuracy: 0.9811 1476/1688 [=========================>....] - ETA: 0s - loss: 0.0614 - accuracy: 0.9810 1503/1688 [=========================>....] - ETA: 0s - loss: 0.0618 - accuracy: 0.9809 1530/1688 [==========================>...] - ETA: 0s - loss: 0.0618 - accuracy: 0.9809 1557/1688 [==========================>...] - ETA: 0s - loss: 0.0622 - accuracy: 0.9808 1584/1688 [===========================>..] - ETA: 0s - loss: 0.0624 - accuracy: 0.9808 1610/1688 [===========================>..] - ETA: 0s - loss: 0.0624 - accuracy: 0.9809 1637/1688 [============================>.] - ETA: 0s - loss: 0.0622 - accuracy: 0.9809 1664/1688 [============================>.] - ETA: 0s - loss: 0.0621 - accuracy: 0.9809 1688/1688 [==============================] - 4s 2ms/step - loss: 0.0621 - accuracy: 0.9809 - val_loss: 0.0574 - val_accuracy: 0.9857 Epoch 6/10 1/1688 [..............................] - ETA: 0s - loss: 0.1171 - accuracy: 0.9375 28/1688 [..............................] - ETA: 3s - loss: 0.0400 - accuracy: 0.9877 55/1688 [..............................] - ETA: 3s - loss: 0.0454 - accuracy: 0.9869 82/1688 [>.............................] - ETA: 3s - loss: 0.0468 - accuracy: 0.9867 109/1688 [>.............................] - ETA: 2s - loss: 0.0475 - accuracy: 0.9865 136/1688 [=>............................] - ETA: 2s - loss: 0.0490 - accuracy: 0.9858 163/1688 [=>............................] - ETA: 2s - loss: 0.0471 - accuracy: 0.9862 190/1688 [==>...........................] - ETA: 2s - loss: 0.0466 - accuracy: 0.9857 217/1688 [==>...........................] - ETA: 2s - loss: 0.0462 - accuracy: 0.9856 244/1688 [===>..........................] - ETA: 2s - loss: 0.0450 - accuracy: 0.9863 271/1688 [===>..........................] - ETA: 2s - loss: 0.0464 - accuracy: 0.9860 298/1688 [====>.........................] - ETA: 2s - loss: 0.0489 - accuracy: 0.9847 325/1688 [====>.........................] - ETA: 2s - loss: 0.0493 - accuracy: 0.9846 352/1688 [=====>........................] - ETA: 2s - loss: 0.0497 - accuracy: 0.9840 379/1688 [=====>........................] - ETA: 2s - loss: 0.0507 - accuracy: 0.9839 406/1688 [======>.......................] - ETA: 2s - loss: 0.0518 - accuracy: 0.9838 433/1688 [======>.......................] - ETA: 2s - loss: 0.0505 - accuracy: 0.9843 460/1688 [=======>......................] - ETA: 2s - loss: 0.0507 - accuracy: 0.9846 487/1688 [=======>......................] - ETA: 2s - loss: 0.0502 - accuracy: 0.9850 514/1688 [========>.....................] - ETA: 2s - loss: 0.0506 - accuracy: 0.9848 541/1688 [========>.....................] - ETA: 2s - loss: 0.0509 - accuracy: 0.9846 568/1688 [=========>....................] - ETA: 2s - loss: 0.0513 - accuracy: 0.9845 595/1688 [=========>....................] - ETA: 2s - loss: 0.0518 - accuracy: 0.9844 621/1688 [==========>...................] - ETA: 2s - loss: 0.0522 - accuracy: 0.9842 648/1688 [==========>...................] - ETA: 1s - loss: 0.0521 - accuracy: 0.9841 675/1688 [==========>...................] - ETA: 1s - loss: 0.0522 - accuracy: 0.9843 702/1688 [===========>..................] - ETA: 1s - loss: 0.0530 - accuracy: 0.9841 729/1688 [===========>..................] - ETA: 1s - loss: 0.0526 - accuracy: 0.9841 755/1688 [============>.................] - ETA: 1s - loss: 0.0524 - accuracy: 0.9843 782/1688 [============>.................] - ETA: 1s - loss: 0.0517 - accuracy: 0.9845 809/1688 [=============>................] - ETA: 1s - loss: 0.0519 - accuracy: 0.9845 835/1688 [=============>................] - ETA: 1s - loss: 0.0520 - accuracy: 0.9845 861/1688 [==============>...............] - ETA: 1s - loss: 0.0524 - accuracy: 0.9842 888/1688 [==============>...............] - ETA: 1s - loss: 0.0518 - accuracy: 0.9844 915/1688 [===============>..............] - ETA: 1s - loss: 0.0517 - accuracy: 0.9843 942/1688 [===============>..............] - ETA: 1s - loss: 0.0517 - accuracy: 0.9842 969/1688 [================>.............] - ETA: 1s - loss: 0.0520 - accuracy: 0.9843 995/1688 [================>.............] - ETA: 1s - loss: 0.0524 - accuracy: 0.9842 1021/1688 [=================>............] - ETA: 1s - loss: 0.0530 - accuracy: 0.9840 1047/1688 [=================>............] - ETA: 1s - loss: 0.0526 - accuracy: 0.9841 1074/1688 [==================>...........] - ETA: 1s - loss: 0.0525 - accuracy: 0.9841 1101/1688 [==================>...........] - ETA: 1s - loss: 0.0520 - accuracy: 0.9843 1128/1688 [===================>..........] - ETA: 1s - loss: 0.0518 - accuracy: 0.9843 1155/1688 [===================>..........] - ETA: 1s - loss: 0.0522 - accuracy: 0.9841 1182/1688 [====================>.........] - ETA: 0s - loss: 0.0532 - accuracy: 0.9840 1208/1688 [====================>.........] - ETA: 0s - loss: 0.0532 - accuracy: 0.9840 1235/1688 [====================>.........] - ETA: 0s - loss: 0.0529 - accuracy: 0.9841 1262/1688 [=====================>........] - ETA: 0s - loss: 0.0530 - accuracy: 0.9841 1288/1688 [=====================>........] - ETA: 0s - loss: 0.0528 - accuracy: 0.9843 1315/1688 [======================>.......] - ETA: 0s - loss: 0.0528 - accuracy: 0.9843 1342/1688 [======================>.......] - ETA: 0s - loss: 0.0532 - accuracy: 0.9842 1369/1688 [=======================>......] - ETA: 0s - loss: 0.0534 - accuracy: 0.9841 1396/1688 [=======================>......] - ETA: 0s - loss: 0.0541 - accuracy: 0.9840 1423/1688 [========================>.....] - ETA: 0s - loss: 0.0544 - accuracy: 0.9839 1450/1688 [========================>.....] - ETA: 0s - loss: 0.0544 - accuracy: 0.9839 1477/1688 [=========================>....] - ETA: 0s - loss: 0.0544 - accuracy: 0.9839 1504/1688 [=========================>....] - ETA: 0s - loss: 0.0546 - accuracy: 0.9839 1531/1688 [==========================>...] - ETA: 0s - loss: 0.0546 - accuracy: 0.9839 1557/1688 [==========================>...] - ETA: 0s - loss: 0.0547 - accuracy: 0.9839 1584/1688 [===========================>..] - ETA: 0s - loss: 0.0547 - accuracy: 0.9839 1611/1688 [===========================>..] - ETA: 0s - loss: 0.0549 - accuracy: 0.9839 1638/1688 [============================>.] - ETA: 0s - loss: 0.0549 - accuracy: 0.9838 1665/1688 [============================>.] - ETA: 0s - loss: 0.0550 - accuracy: 0.9837 1688/1688 [==============================] - 4s 2ms/step - loss: 0.0549 - accuracy: 0.9837 - val_loss: 0.0580 - val_accuracy: 0.9852 Epoch 7/10 1/1688 [..............................] - ETA: 0s - loss: 0.0494 - accuracy: 1.0000 28/1688 [..............................] - ETA: 3s - loss: 0.0314 - accuracy: 0.9911 55/1688 [..............................] - ETA: 3s - loss: 0.0365 - accuracy: 0.9898 81/1688 [>.............................] - ETA: 3s - loss: 0.0427 - accuracy: 0.9884 108/1688 [>.............................] - ETA: 2s - loss: 0.0401 - accuracy: 0.9887 134/1688 [=>............................] - ETA: 2s - loss: 0.0430 - accuracy: 0.9869 161/1688 [=>............................] - ETA: 2s - loss: 0.0417 - accuracy: 0.9866 188/1688 [==>...........................] - ETA: 2s - loss: 0.0441 - accuracy: 0.9867 215/1688 [==>...........................] - ETA: 2s - loss: 0.0448 - accuracy: 0.9865 242/1688 [===>..........................] - ETA: 2s - loss: 0.0446 - accuracy: 0.9863 269/1688 [===>..........................] - ETA: 2s - loss: 0.0474 - accuracy: 0.9851 296/1688 [====>.........................] - ETA: 2s - loss: 0.0467 - accuracy: 0.9851 323/1688 [====>.........................] - ETA: 2s - loss: 0.0462 - accuracy: 0.9856 350/1688 [=====>........................] - ETA: 2s - loss: 0.0455 - accuracy: 0.9860 377/1688 [=====>........................] - ETA: 2s - loss: 0.0458 - accuracy: 0.9857 404/1688 [======>.......................] - ETA: 2s - loss: 0.0470 - accuracy: 0.9857 431/1688 [======>.......................] - ETA: 2s - loss: 0.0466 - accuracy: 0.9860 458/1688 [=======>......................] - ETA: 2s - loss: 0.0475 - accuracy: 0.9856 485/1688 [=======>......................] - ETA: 2s - loss: 0.0470 - accuracy: 0.9858 512/1688 [========>.....................] - ETA: 2s - loss: 0.0471 - accuracy: 0.9860 538/1688 [========>.....................] - ETA: 2s - loss: 0.0475 - accuracy: 0.9858 565/1688 [=========>....................] - ETA: 2s - loss: 0.0478 - accuracy: 0.9858 592/1688 [=========>....................] - ETA: 2s - loss: 0.0483 - accuracy: 0.9858 618/1688 [=========>....................] - ETA: 2s - loss: 0.0487 - accuracy: 0.9858 644/1688 [==========>...................] - ETA: 1s - loss: 0.0486 - accuracy: 0.9858 671/1688 [==========>...................] - ETA: 1s - loss: 0.0487 - accuracy: 0.9858 697/1688 [===========>..................] - ETA: 1s - loss: 0.0488 - accuracy: 0.9857 724/1688 [===========>..................] - ETA: 1s - loss: 0.0487 - accuracy: 0.9857 751/1688 [============>.................] - ETA: 1s - loss: 0.0492 - accuracy: 0.9855 778/1688 [============>.................] - ETA: 1s - loss: 0.0497 - accuracy: 0.9852 805/1688 [=============>................] - ETA: 1s - loss: 0.0496 - accuracy: 0.9852 832/1688 [=============>................] - ETA: 1s - loss: 0.0493 - accuracy: 0.9854 859/1688 [==============>...............] - ETA: 1s - loss: 0.0496 - accuracy: 0.9853 886/1688 [==============>...............] - ETA: 1s - loss: 0.0495 - accuracy: 0.9853 913/1688 [===============>..............] - ETA: 1s - loss: 0.0501 - accuracy: 0.9849 939/1688 [===============>..............] - ETA: 1s - loss: 0.0497 - accuracy: 0.9850 966/1688 [================>.............] - ETA: 1s - loss: 0.0492 - accuracy: 0.9852 993/1688 [================>.............] - ETA: 1s - loss: 0.0493 - accuracy: 0.9853 1019/1688 [=================>............] - ETA: 1s - loss: 0.0491 - accuracy: 0.9854 1045/1688 [=================>............] - ETA: 1s - loss: 0.0490 - accuracy: 0.9854 1072/1688 [==================>...........] - ETA: 1s - loss: 0.0488 - accuracy: 0.9855 1098/1688 [==================>...........] - ETA: 1s - loss: 0.0487 - accuracy: 0.9856 1124/1688 [==================>...........] - ETA: 1s - loss: 0.0486 - accuracy: 0.9855 1151/1688 [===================>..........] - ETA: 1s - loss: 0.0487 - accuracy: 0.9854 1178/1688 [===================>..........] - ETA: 0s - loss: 0.0486 - accuracy: 0.9853 1205/1688 [====================>.........] - ETA: 0s - loss: 0.0484 - accuracy: 0.9854 1232/1688 [====================>.........] - ETA: 0s - loss: 0.0486 - accuracy: 0.9853 1258/1688 [=====================>........] - ETA: 0s - loss: 0.0485 - accuracy: 0.9854 1284/1688 [=====================>........] - ETA: 0s - loss: 0.0489 - accuracy: 0.9853 1311/1688 [======================>.......] - ETA: 0s - loss: 0.0488 - accuracy: 0.9853 1338/1688 [======================>.......] - ETA: 0s - loss: 0.0487 - accuracy: 0.9853 1365/1688 [=======================>......] - ETA: 0s - loss: 0.0485 - accuracy: 0.9854 1392/1688 [=======================>......] - ETA: 0s - loss: 0.0489 - accuracy: 0.9853 1419/1688 [========================>.....] - ETA: 0s - loss: 0.0488 - accuracy: 0.9853 1446/1688 [========================>.....] - ETA: 0s - loss: 0.0490 - accuracy: 0.9851 1473/1688 [=========================>....] - ETA: 0s - loss: 0.0488 - accuracy: 0.9851 1500/1688 [=========================>....] - ETA: 0s - loss: 0.0491 - accuracy: 0.9850 1527/1688 [==========================>...] - ETA: 0s - loss: 0.0493 - accuracy: 0.9849 1554/1688 [==========================>...] - ETA: 0s - loss: 0.0491 - accuracy: 0.9850 1581/1688 [===========================>..] - ETA: 0s - loss: 0.0494 - accuracy: 0.9849 1608/1688 [===========================>..] - ETA: 0s - loss: 0.0495 - accuracy: 0.9847 1635/1688 [============================>.] - ETA: 0s - loss: 0.0493 - accuracy: 0.9847 1662/1688 [============================>.] - ETA: 0s - loss: 0.0494 - accuracy: 0.9848 1688/1688 [==============================] - 4s 2ms/step - loss: 0.0492 - accuracy: 0.9848 - val_loss: 0.0578 - val_accuracy: 0.9840 Epoch 8/10 1/1688 [..............................] - ETA: 0s - loss: 0.0014 - accuracy: 1.0000 28/1688 [..............................] - ETA: 3s - loss: 0.0411 - accuracy: 0.9877 55/1688 [..............................] - ETA: 3s - loss: 0.0364 - accuracy: 0.9892 82/1688 [>.............................] - ETA: 3s - loss: 0.0384 - accuracy: 0.9889 109/1688 [>.............................] - ETA: 2s - loss: 0.0385 - accuracy: 0.9882 136/1688 [=>............................] - ETA: 2s - loss: 0.0374 - accuracy: 0.9890 163/1688 [=>............................] - ETA: 2s - loss: 0.0377 - accuracy: 0.9887 190/1688 [==>...........................] - ETA: 2s - loss: 0.0369 - accuracy: 0.9885 216/1688 [==>...........................] - ETA: 2s - loss: 0.0362 - accuracy: 0.9889 243/1688 [===>..........................] - ETA: 2s - loss: 0.0383 - accuracy: 0.9884 270/1688 [===>..........................] - ETA: 2s - loss: 0.0392 - accuracy: 0.9881 296/1688 [====>.........................] - ETA: 2s - loss: 0.0390 - accuracy: 0.9883 323/1688 [====>.........................] - ETA: 2s - loss: 0.0396 - accuracy: 0.9882 350/1688 [=====>........................] - ETA: 2s - loss: 0.0401 - accuracy: 0.9879 376/1688 [=====>........................] - ETA: 2s - loss: 0.0416 - accuracy: 0.9875 403/1688 [======>.......................] - ETA: 2s - loss: 0.0408 - accuracy: 0.9880 430/1688 [======>.......................] - ETA: 2s - loss: 0.0400 - accuracy: 0.9884 457/1688 [=======>......................] - ETA: 2s - loss: 0.0402 - accuracy: 0.9884 484/1688 [=======>......................] - ETA: 2s - loss: 0.0399 - accuracy: 0.9883 510/1688 [========>.....................] - ETA: 2s - loss: 0.0400 - accuracy: 0.9884 537/1688 [========>.....................] - ETA: 2s - loss: 0.0406 - accuracy: 0.9883 563/1688 [=========>....................] - ETA: 2s - loss: 0.0420 - accuracy: 0.9879 590/1688 [=========>....................] - ETA: 2s - loss: 0.0425 - accuracy: 0.9876 617/1688 [=========>....................] - ETA: 2s - loss: 0.0423 - accuracy: 0.9876 644/1688 [==========>...................] - ETA: 1s - loss: 0.0424 - accuracy: 0.9873 670/1688 [==========>...................] - ETA: 1s - loss: 0.0436 - accuracy: 0.9874 696/1688 [===========>..................] - ETA: 1s - loss: 0.0439 - accuracy: 0.9872 723/1688 [===========>..................] - ETA: 1s - loss: 0.0446 - accuracy: 0.9872 750/1688 [============>.................] - ETA: 1s - loss: 0.0443 - accuracy: 0.9872 776/1688 [============>.................] - ETA: 1s - loss: 0.0443 - accuracy: 0.9872 803/1688 [=============>................] - ETA: 1s - loss: 0.0444 - accuracy: 0.9872 829/1688 [=============>................] - ETA: 1s - loss: 0.0447 - accuracy: 0.9872 855/1688 [==============>...............] - ETA: 1s - loss: 0.0443 - accuracy: 0.9874 881/1688 [==============>...............] - ETA: 1s - loss: 0.0441 - accuracy: 0.9875 908/1688 [===============>..............] - ETA: 1s - loss: 0.0441 - accuracy: 0.9875 935/1688 [===============>..............] - ETA: 1s - loss: 0.0439 - accuracy: 0.9874 962/1688 [================>.............] - ETA: 1s - loss: 0.0440 - accuracy: 0.9873 989/1688 [================>.............] - ETA: 1s - loss: 0.0439 - accuracy: 0.9873 1016/1688 [=================>............] - ETA: 1s - loss: 0.0438 - accuracy: 0.9874 1043/1688 [=================>............] - ETA: 1s - loss: 0.0440 - accuracy: 0.9873 1070/1688 [==================>...........] - ETA: 1s - loss: 0.0440 - accuracy: 0.9872 1097/1688 [==================>...........] - ETA: 1s - loss: 0.0438 - accuracy: 0.9872 1124/1688 [==================>...........] - ETA: 1s - loss: 0.0436 - accuracy: 0.9872 1151/1688 [===================>..........] - ETA: 1s - loss: 0.0436 - accuracy: 0.9872 1178/1688 [===================>..........] - ETA: 0s - loss: 0.0443 - accuracy: 0.9871 1205/1688 [====================>.........] - ETA: 0s - loss: 0.0442 - accuracy: 0.9871 1232/1688 [====================>.........] - ETA: 0s - loss: 0.0440 - accuracy: 0.9872 1259/1688 [=====================>........] - ETA: 0s - loss: 0.0441 - accuracy: 0.9871 1286/1688 [=====================>........] - ETA: 0s - loss: 0.0440 - accuracy: 0.9871 1313/1688 [======================>.......] - ETA: 0s - loss: 0.0439 - accuracy: 0.9871 1339/1688 [======================>.......] - ETA: 0s - loss: 0.0438 - accuracy: 0.9871 1366/1688 [=======================>......] - ETA: 0s - loss: 0.0437 - accuracy: 0.9871 1393/1688 [=======================>......] - ETA: 0s - loss: 0.0441 - accuracy: 0.9871 1420/1688 [========================>.....] - ETA: 0s - loss: 0.0437 - accuracy: 0.9872 1447/1688 [========================>.....] - ETA: 0s - loss: 0.0435 - accuracy: 0.9872 1474/1688 [=========================>....] - ETA: 0s - loss: 0.0433 - accuracy: 0.9872 1500/1688 [=========================>....] - ETA: 0s - loss: 0.0435 - accuracy: 0.9872 1526/1688 [==========================>...] - ETA: 0s - loss: 0.0437 - accuracy: 0.9871 1553/1688 [==========================>...] - ETA: 0s - loss: 0.0438 - accuracy: 0.9870 1579/1688 [===========================>..] - ETA: 0s - loss: 0.0438 - accuracy: 0.9870 1605/1688 [===========================>..] - ETA: 0s - loss: 0.0439 - accuracy: 0.9870 1632/1688 [============================>.] - ETA: 0s - loss: 0.0438 - accuracy: 0.9870 1659/1688 [============================>.] - ETA: 0s - loss: 0.0438 - accuracy: 0.9870 1686/1688 [============================>.] - ETA: 0s - loss: 0.0439 - accuracy: 0.9870 1688/1688 [==============================] - 4s 2ms/step - loss: 0.0440 - accuracy: 0.9869 - val_loss: 0.0614 - val_accuracy: 0.9833 Epoch 9/10 1/1688 [..............................] - ETA: 0s - loss: 0.0323 - accuracy: 1.0000 28/1688 [..............................] - ETA: 3s - loss: 0.0494 - accuracy: 0.9866 55/1688 [..............................] - ETA: 3s - loss: 0.0385 - accuracy: 0.9903 81/1688 [>.............................] - ETA: 3s - loss: 0.0392 - accuracy: 0.9900 108/1688 [>.............................] - ETA: 2s - loss: 0.0393 - accuracy: 0.9902 135/1688 [=>............................] - ETA: 2s - loss: 0.0407 - accuracy: 0.9898 162/1688 [=>............................] - ETA: 2s - loss: 0.0426 - accuracy: 0.9882 188/1688 [==>...........................] - ETA: 2s - loss: 0.0435 - accuracy: 0.9880 215/1688 [==>...........................] - ETA: 2s - loss: 0.0441 - accuracy: 0.9876 242/1688 [===>..........................] - ETA: 2s - loss: 0.0442 - accuracy: 0.9877 269/1688 [===>..........................] - ETA: 2s - loss: 0.0444 - accuracy: 0.9873 296/1688 [====>.........................] - ETA: 2s - loss: 0.0423 - accuracy: 0.9879 323/1688 [====>.........................] - ETA: 2s - loss: 0.0414 - accuracy: 0.9878 350/1688 [=====>........................] - ETA: 2s - loss: 0.0411 - accuracy: 0.9880 377/1688 [=====>........................] - ETA: 2s - loss: 0.0401 - accuracy: 0.9884 404/1688 [======>.......................] - ETA: 2s - loss: 0.0404 - accuracy: 0.9882 431/1688 [======>.......................] - ETA: 2s - loss: 0.0400 - accuracy: 0.9881 458/1688 [=======>......................] - ETA: 2s - loss: 0.0405 - accuracy: 0.9880 485/1688 [=======>......................] - ETA: 2s - loss: 0.0400 - accuracy: 0.9881 512/1688 [========>.....................] - ETA: 2s - loss: 0.0398 - accuracy: 0.9883 539/1688 [========>.....................] - ETA: 2s - loss: 0.0401 - accuracy: 0.9882 566/1688 [=========>....................] - ETA: 2s - loss: 0.0411 - accuracy: 0.9877 593/1688 [=========>....................] - ETA: 2s - loss: 0.0408 - accuracy: 0.9878 620/1688 [==========>...................] - ETA: 2s - loss: 0.0409 - accuracy: 0.9877 647/1688 [==========>...................] - ETA: 1s - loss: 0.0404 - accuracy: 0.9880 674/1688 [==========>...................] - ETA: 1s - loss: 0.0402 - accuracy: 0.9880 701/1688 [===========>..................] - ETA: 1s - loss: 0.0409 - accuracy: 0.9879 727/1688 [===========>..................] - ETA: 1s - loss: 0.0418 - accuracy: 0.9877 754/1688 [============>.................] - ETA: 1s - loss: 0.0421 - accuracy: 0.9877 781/1688 [============>.................] - ETA: 1s - loss: 0.0424 - accuracy: 0.9876 808/1688 [=============>................] - ETA: 1s - loss: 0.0426 - accuracy: 0.9875 835/1688 [=============>................] - ETA: 1s - loss: 0.0429 - accuracy: 0.9875 862/1688 [==============>...............] - ETA: 1s - loss: 0.0426 - accuracy: 0.9875 888/1688 [==============>...............] - ETA: 1s - loss: 0.0427 - accuracy: 0.9874 915/1688 [===============>..............] - ETA: 1s - loss: 0.0429 - accuracy: 0.9873 942/1688 [===============>..............] - ETA: 1s - loss: 0.0431 - accuracy: 0.9872 969/1688 [================>.............] - ETA: 1s - loss: 0.0433 - accuracy: 0.9871 995/1688 [================>.............] - ETA: 1s - loss: 0.0435 - accuracy: 0.9870 1022/1688 [=================>............] - ETA: 1s - loss: 0.0434 - accuracy: 0.9871 1049/1688 [=================>............] - ETA: 1s - loss: 0.0437 - accuracy: 0.9870 1075/1688 [==================>...........] - ETA: 1s - loss: 0.0432 - accuracy: 0.9872 1102/1688 [==================>...........] - ETA: 1s - loss: 0.0426 - accuracy: 0.9874 1128/1688 [===================>..........] - ETA: 1s - loss: 0.0424 - accuracy: 0.9875 1154/1688 [===================>..........] - ETA: 1s - loss: 0.0427 - accuracy: 0.9873 1181/1688 [===================>..........] - ETA: 0s - loss: 0.0426 - accuracy: 0.9874 1208/1688 [====================>.........] - ETA: 0s - loss: 0.0423 - accuracy: 0.9874 1234/1688 [====================>.........] - ETA: 0s - loss: 0.0424 - accuracy: 0.9873 1261/1688 [=====================>........] - ETA: 0s - loss: 0.0425 - accuracy: 0.9872 1288/1688 [=====================>........] - ETA: 0s - loss: 0.0424 - accuracy: 0.9872 1315/1688 [======================>.......] - ETA: 0s - loss: 0.0425 - accuracy: 0.9870 1342/1688 [======================>.......] - ETA: 0s - loss: 0.0422 - accuracy: 0.9871 1369/1688 [=======================>......] - ETA: 0s - loss: 0.0420 - accuracy: 0.9871 1395/1688 [=======================>......] - ETA: 0s - loss: 0.0419 - accuracy: 0.9871 1422/1688 [========================>.....] - ETA: 0s - loss: 0.0418 - accuracy: 0.9871 1449/1688 [========================>.....] - ETA: 0s - loss: 0.0419 - accuracy: 0.9871 1476/1688 [=========================>....] - ETA: 0s - loss: 0.0418 - accuracy: 0.9871 1503/1688 [=========================>....] - ETA: 0s - loss: 0.0415 - accuracy: 0.9873 1530/1688 [==========================>...] - ETA: 0s - loss: 0.0414 - accuracy: 0.9872 1557/1688 [==========================>...] - ETA: 0s - loss: 0.0412 - accuracy: 0.9874 1584/1688 [===========================>..] - ETA: 0s - loss: 0.0412 - accuracy: 0.9873 1611/1688 [===========================>..] - ETA: 0s - loss: 0.0410 - accuracy: 0.9873 1638/1688 [============================>.] - ETA: 0s - loss: 0.0410 - accuracy: 0.9873 1665/1688 [============================>.] - ETA: 0s - loss: 0.0412 - accuracy: 0.9871 1688/1688 [==============================] - 4s 2ms/step - loss: 0.0412 - accuracy: 0.9871 - val_loss: 0.0548 - val_accuracy: 0.9857 Epoch 10/10 1/1688 [..............................] - ETA: 0s - loss: 0.1887 - accuracy: 0.9688 28/1688 [..............................] - ETA: 3s - loss: 0.0339 - accuracy: 0.9933 55/1688 [..............................] - ETA: 3s - loss: 0.0306 - accuracy: 0.9926 82/1688 [>.............................] - ETA: 3s - loss: 0.0291 - accuracy: 0.9924 108/1688 [>.............................] - ETA: 2s - loss: 0.0307 - accuracy: 0.9902 135/1688 [=>............................] - ETA: 2s - loss: 0.0313 - accuracy: 0.9905 162/1688 [=>............................] - ETA: 2s - loss: 0.0336 - accuracy: 0.9907 188/1688 [==>...........................] - ETA: 2s - loss: 0.0327 - accuracy: 0.9915 215/1688 [==>...........................] - ETA: 2s - loss: 0.0341 - accuracy: 0.9913 241/1688 [===>..........................] - ETA: 2s - loss: 0.0348 - accuracy: 0.9911 268/1688 [===>..........................] - ETA: 2s - loss: 0.0368 - accuracy: 0.9895 295/1688 [====>.........................] - ETA: 2s - loss: 0.0374 - accuracy: 0.9891 322/1688 [====>.........................] - ETA: 2s - loss: 0.0377 - accuracy: 0.9891 349/1688 [=====>........................] - ETA: 2s - loss: 0.0367 - accuracy: 0.9895 376/1688 [=====>........................] - ETA: 2s - loss: 0.0366 - accuracy: 0.9897 403/1688 [======>.......................] - ETA: 2s - loss: 0.0358 - accuracy: 0.9899 429/1688 [======>.......................] - ETA: 2s - loss: 0.0356 - accuracy: 0.9899 455/1688 [=======>......................] - ETA: 2s - loss: 0.0358 - accuracy: 0.9898 482/1688 [=======>......................] - ETA: 2s - loss: 0.0361 - accuracy: 0.9895 509/1688 [========>.....................] - ETA: 2s - loss: 0.0358 - accuracy: 0.9895 536/1688 [========>.....................] - ETA: 2s - loss: 0.0356 - accuracy: 0.9894 562/1688 [========>.....................] - ETA: 2s - loss: 0.0352 - accuracy: 0.9896 588/1688 [=========>....................] - ETA: 2s - loss: 0.0362 - accuracy: 0.9894 614/1688 [=========>....................] - ETA: 2s - loss: 0.0362 - accuracy: 0.9893 641/1688 [==========>...................] - ETA: 1s - loss: 0.0369 - accuracy: 0.9891 667/1688 [==========>...................] - ETA: 1s - loss: 0.0369 - accuracy: 0.9892 694/1688 [===========>..................] - ETA: 1s - loss: 0.0372 - accuracy: 0.9891 720/1688 [===========>..................] - ETA: 1s - loss: 0.0372 - accuracy: 0.9891 747/1688 [============>.................] - ETA: 1s - loss: 0.0370 - accuracy: 0.9893 774/1688 [============>.................] - ETA: 1s - loss: 0.0371 - accuracy: 0.9893 801/1688 [=============>................] - ETA: 1s - loss: 0.0370 - accuracy: 0.9894 828/1688 [=============>................] - ETA: 1s - loss: 0.0373 - accuracy: 0.9891 854/1688 [==============>...............] - ETA: 1s - loss: 0.0371 - accuracy: 0.9892 881/1688 [==============>...............] - ETA: 1s - loss: 0.0372 - accuracy: 0.9891 908/1688 [===============>..............] - ETA: 1s - loss: 0.0368 - accuracy: 0.9892 935/1688 [===============>..............] - ETA: 1s - loss: 0.0366 - accuracy: 0.9892 962/1688 [================>.............] - ETA: 1s - loss: 0.0369 - accuracy: 0.9892 989/1688 [================>.............] - ETA: 1s - loss: 0.0369 - accuracy: 0.9893 1016/1688 [=================>............] - ETA: 1s - loss: 0.0369 - accuracy: 0.9893 1042/1688 [=================>............] - ETA: 1s - loss: 0.0369 - accuracy: 0.9893 1069/1688 [=================>............] - ETA: 1s - loss: 0.0366 - accuracy: 0.9894 1095/1688 [==================>...........] - ETA: 1s - loss: 0.0367 - accuracy: 0.9894 1122/1688 [==================>...........] - ETA: 1s - loss: 0.0369 - accuracy: 0.9893 1149/1688 [===================>..........] - ETA: 1s - loss: 0.0369 - accuracy: 0.9892 1175/1688 [===================>..........] - ETA: 0s - loss: 0.0371 - accuracy: 0.9893 1201/1688 [====================>.........] - ETA: 0s - loss: 0.0368 - accuracy: 0.9893 1228/1688 [====================>.........] - ETA: 0s - loss: 0.0369 - accuracy: 0.9893 1255/1688 [=====================>........] - ETA: 0s - loss: 0.0370 - accuracy: 0.9891 1281/1688 [=====================>........] - ETA: 0s - loss: 0.0370 - accuracy: 0.9892 1308/1688 [======================>.......] - ETA: 0s - loss: 0.0368 - accuracy: 0.9892 1335/1688 [======================>.......] - ETA: 0s - loss: 0.0372 - accuracy: 0.9891 1362/1688 [=======================>......] - ETA: 0s - loss: 0.0371 - accuracy: 0.9891 1388/1688 [=======================>......] - ETA: 0s - loss: 0.0370 - accuracy: 0.9891 1415/1688 [========================>.....] - ETA: 0s - loss: 0.0371 - accuracy: 0.9890 1441/1688 [========================>.....] - ETA: 0s - loss: 0.0373 - accuracy: 0.9890 1467/1688 [=========================>....] - ETA: 0s - loss: 0.0373 - accuracy: 0.9889 1494/1688 [=========================>....] - ETA: 0s - loss: 0.0373 - accuracy: 0.9889 1520/1688 [==========================>...] - ETA: 0s - loss: 0.0371 - accuracy: 0.9889 1547/1688 [==========================>...] - ETA: 0s - loss: 0.0371 - accuracy: 0.9889 1574/1688 [==========================>...] - ETA: 0s - loss: 0.0373 - accuracy: 0.9888 1601/1688 [===========================>..] - ETA: 0s - loss: 0.0370 - accuracy: 0.9889 1628/1688 [===========================>..] - ETA: 0s - loss: 0.0374 - accuracy: 0.9888 1655/1688 [============================>.] - ETA: 0s - loss: 0.0376 - accuracy: 0.9887 1681/1688 [============================>.] - ETA: 0s - loss: 0.0375 - accuracy: 0.9887 1688/1688 [==============================] - 4s 2ms/step - loss: 0.0375 - accuracy: 0.9887 - val_loss: 0.0577 - val_accuracy: 0.9855 <tensorflow.python.keras.callbacks.History at 0x7f3dcbe7d588>
评估基准模型并保存以备稍后使用
_, baseline_model_accuracy = model.evaluate(
test_images, test_labels, verbose=0)
print('Baseline test accuracy:', baseline_model_accuracy)
_, keras_file = tempfile.mkstemp('.h5')
print('Saving model to: ', keras_file)
tf.keras.models.save_model(model, keras_file, include_optimizer=False)
Baseline test accuracy: 0.9807999730110168 Saving model to: /tmp/tmpkenu8pu1.h5
通过聚类微调预训练模型
将 cluster_weights() API 应用于整个预训练模型，以演示它不仅能够在应用 zip 后有效缩减模型大小，还能保持良好的准确率。有关如何以最佳方式平衡用例的准确率和压缩率，请参阅综合指南中的每层示例。
定义模型并应用聚类 API
在将模型传递给聚类 API 之前，请确保它已经过训练并表现出可接受的准确率。
import tensorflow_model_optimization as tfmot
cluster_weights = tfmot.clustering.keras.cluster_weights
CentroidInitialization = tfmot.clustering.keras.CentroidInitialization
clustering_params = {
'number_of_clusters': 16,
'cluster_centroids_init': CentroidInitialization.LINEAR
}
# Cluster a whole model
clustered_model = cluster_weights(model, **clustering_params)
# Use smaller learning rate for fine-tuning clustered model
opt = tf.keras.optimizers.Adam(learning_rate=1e-5)
clustered_model.compile(
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=opt,
metrics=['accuracy'])
clustered_model.summary()
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= cluster_reshape (ClusterWeig (None, 28, 28, 1) 0 _________________________________________________________________ cluster_conv2d (ClusterWeigh (None, 26, 26, 12) 136 _________________________________________________________________ cluster_max_pooling2d (Clust (None, 13, 13, 12) 0 _________________________________________________________________ cluster_flatten (ClusterWeig (None, 2028) 0 _________________________________________________________________ cluster_dense (ClusterWeight (None, 10) 20306 ================================================================= Total params: 20,442 Trainable params: 54 Non-trainable params: 20,388 _________________________________________________________________
微调模型并根据基准评估准确率
使用聚类对模型进行 1 个周期的微调。
# Fine-tune model
clustered_model.fit(
train_images,
train_labels,
batch_size=500,
epochs=1,
validation_split=0.1)
1/108 [..............................] - ETA: 0s - loss: 0.0434 - accuracy: 0.9840 19/108 [====>.........................] - ETA: 0s - loss: 0.0566 - accuracy: 0.9805 36/108 [=========>....................] - ETA: 0s - loss: 0.0556 - accuracy: 0.9803 54/108 [==============>...............] - ETA: 0s - loss: 0.0537 - accuracy: 0.9814 72/108 [===================>..........] - ETA: 0s - loss: 0.0556 - accuracy: 0.9806 90/108 [========================>.....] - ETA: 0s - loss: 0.0556 - accuracy: 0.9808 108/108 [==============================] - ETA: 0s - loss: 0.0547 - accuracy: 0.9807 108/108 [==============================] - 0s 4ms/step - loss: 0.0547 - accuracy: 0.9807 - val_loss: 0.0804 - val_accuracy: 0.9760 <tensorflow.python.keras.callbacks.History at 0x7f3e4116ab70>
对于本示例，与基准相比，聚类后的测试准确率损失最小。
_, clustered_model_accuracy = clustered_model.evaluate(
test_images, test_labels, verbose=0)
print('Baseline test accuracy:', baseline_model_accuracy)
print('Clustered test accuracy:', clustered_model_accuracy)
Baseline test accuracy: 0.9807999730110168 Clustered test accuracy: 0.9760000109672546
通过聚类创建大小缩减至六分之一的模型
strip_clustering 和应用标准压缩算法（例如通过 gzip）对于看到聚类压缩的好处必不可少。
首先，为 TensorFlow 创建一个可压缩模型。在这里，strip_clustering 会移除聚类仅在训练期间才需要的所有变量（例如用于存储簇形心和索引的 tf.Variable），否则这些变量会在推理期间增加模型大小。
final_model = tfmot.clustering.keras.strip_clustering(clustered_model)
_, clustered_keras_file = tempfile.mkstemp('.h5')
print('Saving clustered model to: ', clustered_keras_file)
tf.keras.models.save_model(final_model, clustered_keras_file,
include_optimizer=False)
Saving clustered model to: /tmp/tmpsc3jb7v8.h5
随后，为 TFLite 创建可压缩模型。您可以将聚类模型转换为可在目标后端上运行的格式。TensorFlow Lite 是可用于部署到移动设备的示例。
clustered_tflite_file = '/tmp/clustered_mnist.tflite'
converter = tf.lite.TFLiteConverter.from_keras_model(final_model)
tflite_clustered_model = converter.convert()
with open(clustered_tflite_file, 'wb') as f:
f.write(tflite_clustered_model)
print('Saved clustered TFLite model to:', clustered_tflite_file)
WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/training/tracking/tracking.py:111: Model.state_updates (from tensorflow.python.keras.engine.training) is deprecated and will be removed in a future version. Instructions for updating: This property should not be used in TensorFlow 2.0, as updates are applied automatically. WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.6/site-packages/tensorflow/python/training/tracking/tracking.py:111: Layer.updates (from tensorflow.python.keras.engine.base_layer) is deprecated and will be removed in a future version. Instructions for updating: This property should not be used in TensorFlow 2.0, as updates are applied automatically. INFO:tensorflow:Assets written to: /tmp/tmp69qei5fh/assets Saved clustered TFLite model to: /tmp/clustered_mnist.tflite
定义一个辅助函数，通过 gzip 实际压缩模型并测量压缩后的大小。
def get_gzipped_model_size(file):
# It returns the size of the gzipped model in bytes.
import os
import zipfile
_, zipped_file = tempfile.mkstemp('.zip')
with zipfile.ZipFile(zipped_file, 'w', compression=zipfile.ZIP_DEFLATED) as f:
f.write(file)
return os.path.getsize(zipped_file)
比较后可以发现，聚类使模型大小缩减至原来的六分之一
print("Size of gzipped baseline Keras model: %.2f bytes" % (get_gzipped_model_size(keras_file)))
print("Size of gzipped clustered Keras model: %.2f bytes" % (get_gzipped_model_size(clustered_keras_file)))
print("Size of gzipped clustered TFlite model: %.2f bytes" % (get_gzipped_model_size(clustered_tflite_file)))
Size of gzipped baseline Keras model: 78076.00 bytes Size of gzipped clustered Keras model: 12728.00 bytes Size of gzipped clustered TFlite model: 12126.00 bytes
通过将权重聚类与训练后量化相结合，创建一个大小缩减至八分之一的 TFLite 模型
您可以将训练后量化应用于聚类模型来获得更多好处。
converter = tf.lite.TFLiteConverter.from_keras_model(final_model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_quant_model = converter.convert()
_, quantized_and_clustered_tflite_file = tempfile.mkstemp('.tflite')
with open(quantized_and_clustered_tflite_file, 'wb') as f:
f.write(tflite_quant_model)
print('Saved quantized and clustered TFLite model to:', quantized_and_clustered_tflite_file)
print("Size of gzipped baseline Keras model: %.2f bytes" % (get_gzipped_model_size(keras_file)))
print("Size of gzipped clustered and quantized TFlite model: %.2f bytes" % (get_gzipped_model_size(quantized_and_clustered_tflite_file)))
INFO:tensorflow:Assets written to: /tmp/tmpmzv1zby7/assets INFO:tensorflow:Assets written to: /tmp/tmpmzv1zby7/assets Saved quantized and clustered TFLite model to: /tmp/tmp5yu2mobb.tflite Size of gzipped baseline Keras model: 78076.00 bytes Size of gzipped clustered and quantized TFlite model: 9237.00 bytes
查看从 TF 到 TFLite 的准确率持久性
定义一个辅助函数，基于测试数据集评估 TFLite 模型。
def eval_model(interpreter):
input_index = interpreter.get_input_details()[0]["index"]
output_index = interpreter.get_output_details()[0]["index"]
# Run predictions on every image in the "test" dataset.
prediction_digits = []
for i, test_image in enumerate(test_images):
if i % 1000 == 0:
print('Evaluated on {n} results so far.'.format(n=i))
# Pre-processing: add batch dimension and convert to float32 to match with
# the model's input data format.
test_image = np.expand_dims(test_image, axis=0).astype(np.float32)
interpreter.set_tensor(input_index, test_image)
# Run inference.
interpreter.invoke()
# Post-processing: remove batch dimension and find the digit with highest
# probability.
output = interpreter.tensor(output_index)
digit = np.argmax(output()[0])
prediction_digits.append(digit)
print('\n')
# Compare prediction results with ground truth labels to calculate accuracy.
prediction_digits = np.array(prediction_digits)
accuracy = (prediction_digits == test_labels).mean()
return accuracy
评估已被聚类和量化的模型后，您将看到从 TensorFlow 持续到 TFLite 后端的准确率。
interpreter = tf.lite.Interpreter(model_content=tflite_quant_model)
interpreter.allocate_tensors()
test_accuracy = eval_model(interpreter)
print('Clustered and quantized TFLite test_accuracy:', test_accuracy)
print('Clustered TF test accuracy:', clustered_model_accuracy)
Evaluated on 0 results so far. Evaluated on 1000 results so far. Evaluated on 2000 results so far. Evaluated on 3000 results so far. Evaluated on 4000 results so far. Evaluated on 5000 results so far. Evaluated on 6000 results so far. Evaluated on 7000 results so far. Evaluated on 8000 results so far. Evaluated on 9000 results so far. Clustered and quantized TFLite test_accuracy: 0.9759 Clustered TF test accuracy: 0.9760000109672546
结论
在本教程中，您了解了如何使用 TensorFlow Model Optimization Toolkit API 创建聚类模型。更具体地说，您已经从头至尾完成了一个端到端示例，此示例为 MNIST 创建了一个大小缩减至原来的八分之一且准确率差异最小的模型。我们鼓励您试用这项新功能，这对于在资源受限的环境中进行部署特别重要。