 TensorFlow.org에서 보기 |
 Run in Google Colab |
 View source on GitHub |
 노트북 다운로드하기 |
개요
TensorFlow 모델 최적화 도구 키트의 일부인 가중치 클러스터링에 대한 엔드 투 엔드 예제를 소개합니다.
기타 페이지
가중치 클러스터링에 대한 소개와 이를 사용해야 하는지 여부(지원 내용 포함)를 결정하려면 개요 페이지를 참조하세요.
16개의 클러스터로 모델을 완전하게 클러스터링하는 등 해당 사용 사례에 필요한 API를 빠르게 찾으려면 종합 가이드를 참조하세요.
내용
이 튜토리얼에서는 다음을 수행합니다.
- MNIST 데이터세트를 위한
tf.keras모델을 처음부터 훈련합니다. - 가중치 클러스터링 API를 적용하여 모델을 미세 조정하고 정확성을 확인합니다.
- 클러스터링으로부터 6배 더 작은 TF 및 TFLite 모델을 만듭니다.
- 가중치 클러스터링과 훈련 후 양자화를 결합하여 8배 더 작은 TFLite 모델을 만듭니다.
- TF에서 TFLite로 정확성이 지속되는지 확인합니다.
설정
이 Jupyter 노트북은 로컬 virtualenv 또는 colab에서 실행할 수 있습니다. 종속성 설정에 대한 자세한 내용은 설치 가이드를 참조하세요.
pip install -q tensorflow-model-optimizationimport tensorflow as tf
from tensorflow import keras
import numpy as np
import tempfile
import zipfile
import os
클러스터링을 사용하지 않고 MNIST용 tf.keras 모델 훈련하기
# Load MNIST dataset
mnist = keras.datasets.mnist
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()
# Normalize the input image so that each pixel value is between 0 to 1.
train_images = train_images / 255.0
test_images = test_images / 255.0
# Define the model architecture.
model = keras.Sequential([
keras.layers.InputLayer(input_shape=(28, 28)),
keras.layers.Reshape(target_shape=(28, 28, 1)),
keras.layers.Conv2D(filters=12, kernel_size=(3, 3), activation=tf.nn.relu),
keras.layers.MaxPooling2D(pool_size=(2, 2)),
keras.layers.Flatten(),
keras.layers.Dense(10)
])
# Train the digit classification model
model.compile(optimizer='adam',
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
model.fit(
train_images,
train_labels,
validation_split=0.1,
epochs=10
)
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz 8192/11490434 [..............................] - ETA: 0s 5521408/11490434 [=============>................] - ETA: 0s 11141120/11490434 [============================>.] - ETA: 0s 11493376/11490434 [==============================] - 0s 0us/step Epoch 1/10 1/1688 [..............................] - ETA: 1:08:03 - loss: 2.3207 - accuracy: 0.0938 24/1688 [..............................] - ETA: 3s - loss: 2.1736 - accuracy: 0.2709 48/1688 [..............................] - ETA: 3s - loss: 2.0040 - accuracy: 0.3970 70/1688 [>.............................] - ETA: 3s - loss: 1.8520 - accuracy: 0.4644 92/1688 [>.............................] - ETA: 3s - loss: 1.7180 - accuracy: 0.5137 114/1688 [=>............................] - ETA: 3s - loss: 1.6052 - accuracy: 0.5514 136/1688 [=>............................] - ETA: 3s - loss: 1.5111 - accuracy: 0.5811 158/1688 [=>............................] - ETA: 3s - loss: 1.4311 - accuracy: 0.6054 180/1688 [==>...........................] - ETA: 3s - loss: 1.3624 - accuracy: 0.6257 203/1688 [==>...........................] - ETA: 3s - loss: 1.3007 - accuracy: 0.6434 226/1688 [===>..........................] - ETA: 3s - loss: 1.2469 - accuracy: 0.6587 247/1688 [===>..........................] - ETA: 3s - loss: 1.2031 - accuracy: 0.6710 269/1688 [===>..........................] - ETA: 3s - loss: 1.1623 - accuracy: 0.6823 291/1688 [====>.........................] - ETA: 3s - loss: 1.1257 - accuracy: 0.6924 314/1688 [====>.........................] - ETA: 3s - loss: 1.0914 - accuracy: 0.7018 336/1688 [====>.........................] - ETA: 3s - loss: 1.0616 - accuracy: 0.7100 359/1688 [=====>........................] - ETA: 3s - loss: 1.0331 - accuracy: 0.7178 382/1688 [=====>........................] - ETA: 2s - loss: 1.0072 - accuracy: 0.7249 405/1688 [======>.......................] - ETA: 2s - loss: 0.9833 - accuracy: 0.7313 427/1688 [======>.......................] - ETA: 2s - loss: 0.9622 - accuracy: 0.7371 449/1688 [======>.......................] - ETA: 2s - loss: 0.9424 - accuracy: 0.7424 471/1688 [=======>......................] - ETA: 2s - loss: 0.9238 - accuracy: 0.7474 494/1688 [=======>......................] - ETA: 2s - loss: 0.9057 - accuracy: 0.7523 516/1688 [========>.....................] - ETA: 2s - loss: 0.8895 - accuracy: 0.7566 538/1688 [========>.....................] - ETA: 2s - loss: 0.8743 - accuracy: 0.7607 560/1688 [========>.....................] - ETA: 2s - loss: 0.8598 - accuracy: 0.7645 582/1688 [=========>....................] - ETA: 2s - loss: 0.8461 - accuracy: 0.7682 604/1688 [=========>....................] - ETA: 2s - loss: 0.8331 - accuracy: 0.7716 627/1688 [==========>...................] - ETA: 2s - loss: 0.8202 - accuracy: 0.7750 650/1688 [==========>...................] - ETA: 2s - loss: 0.8080 - accuracy: 0.7782 672/1688 [==========>...................] - ETA: 2s - loss: 0.7970 - accuracy: 0.7811 694/1688 [===========>..................] - ETA: 2s - loss: 0.7865 - accuracy: 0.7839 716/1688 [===========>..................] - ETA: 2s - loss: 0.7764 - accuracy: 0.7866 738/1688 [============>.................] - ETA: 2s - loss: 0.7668 - accuracy: 0.7891 760/1688 [============>.................] - ETA: 2s - loss: 0.7575 - accuracy: 0.7916 782/1688 [============>.................] - ETA: 2s - loss: 0.7485 - accuracy: 0.7940 805/1688 [=============>................] - ETA: 2s - loss: 0.7395 - accuracy: 0.7964 828/1688 [=============>................] - ETA: 1s - loss: 0.7308 - accuracy: 0.7987 851/1688 [==============>...............] - ETA: 1s - loss: 0.7225 - accuracy: 0.8009 873/1688 [==============>...............] - ETA: 1s - loss: 0.7148 - accuracy: 0.8029 894/1688 [==============>...............] - ETA: 1s - loss: 0.7077 - accuracy: 0.8048 915/1688 [===============>..............] - ETA: 1s - loss: 0.7008 - accuracy: 0.8066 937/1688 [===============>..............] - ETA: 1s - loss: 0.6939 - accuracy: 0.8085 959/1688 [================>.............] - ETA: 1s - loss: 0.6872 - accuracy: 0.8102 981/1688 [================>.............] - ETA: 1s - loss: 0.6808 - accuracy: 0.8120 1003/1688 [================>.............] - ETA: 1s - loss: 0.6745 - accuracy: 0.8136 1026/1688 [=================>............] - ETA: 1s - loss: 0.6682 - accuracy: 0.8153 1048/1688 [=================>............] - ETA: 1s - loss: 0.6623 - accuracy: 0.8169 1070/1688 [==================>...........] - ETA: 1s - loss: 0.6566 - accuracy: 0.8184 1093/1688 [==================>...........] - ETA: 1s - loss: 0.6507 - accuracy: 0.8200 1116/1688 [==================>...........] - ETA: 1s - loss: 0.6451 - accuracy: 0.8215 1138/1688 [===================>..........] - ETA: 1s - loss: 0.6398 - accuracy: 0.8229 1160/1688 [===================>..........] - ETA: 1s - loss: 0.6347 - accuracy: 0.8243 1183/1688 [====================>.........] - ETA: 1s - loss: 0.6296 - accuracy: 0.8257 1205/1688 [====================>.........] - ETA: 1s - loss: 0.6247 - accuracy: 0.8270 1227/1688 [====================>.........] - ETA: 1s - loss: 0.6200 - accuracy: 0.8282 1249/1688 [=====================>........] - ETA: 1s - loss: 0.6154 - accuracy: 0.8295 1271/1688 [=====================>........] - ETA: 0s - loss: 0.6110 - accuracy: 0.8307 1294/1688 [=====================>........] - ETA: 0s - loss: 0.6064 - accuracy: 0.8319 1316/1688 [======================>.......] - ETA: 0s - loss: 0.6021 - accuracy: 0.8331 1338/1688 [======================>.......] - ETA: 0s - loss: 0.5979 - accuracy: 0.8342 1361/1688 [=======================>......] - ETA: 0s - loss: 0.5936 - accuracy: 0.8353 1382/1688 [=======================>......] - ETA: 0s - loss: 0.5897 - accuracy: 0.8364 1405/1688 [=======================>......] - ETA: 0s - loss: 0.5856 - accuracy: 0.8375 1428/1688 [========================>.....] - ETA: 0s - loss: 0.5816 - accuracy: 0.8386 1451/1688 [========================>.....] - ETA: 0s - loss: 0.5777 - accuracy: 0.8396 1473/1688 [=========================>....] - ETA: 0s - loss: 0.5740 - accuracy: 0.8406 1495/1688 [=========================>....] - ETA: 0s - loss: 0.5704 - accuracy: 0.8416 1517/1688 [=========================>....] - ETA: 0s - loss: 0.5668 - accuracy: 0.8426 1539/1688 [==========================>...] - ETA: 0s - loss: 0.5634 - accuracy: 0.8435 1561/1688 [==========================>...] - ETA: 0s - loss: 0.5600 - accuracy: 0.8444 1583/1688 [===========================>..] - ETA: 0s - loss: 0.5566 - accuracy: 0.8453 1606/1688 [===========================>..] - ETA: 0s - loss: 0.5532 - accuracy: 0.8462 1629/1688 [===========================>..] - ETA: 0s - loss: 0.5498 - accuracy: 0.8472 1652/1688 [============================>.] - ETA: 0s - loss: 0.5465 - accuracy: 0.8481 1674/1688 [============================>.] - ETA: 0s - loss: 0.5434 - accuracy: 0.8489 1688/1688 [==============================] - 7s 3ms/step - loss: 0.5413 - accuracy: 0.8495 - val_loss: 0.1231 - val_accuracy: 0.9673 Epoch 2/10 1/1688 [..............................] - ETA: 5s - loss: 0.0882 - accuracy: 0.9688 25/1688 [..............................] - ETA: 3s - loss: 0.1154 - accuracy: 0.9631 49/1688 [..............................] - ETA: 3s - loss: 0.1263 - accuracy: 0.9597 73/1688 [>.............................] - ETA: 3s - loss: 0.1304 - accuracy: 0.9583 96/1688 [>.............................] - ETA: 3s - loss: 0.1335 - accuracy: 0.9573 120/1688 [=>............................] - ETA: 3s - loss: 0.1357 - accuracy: 0.9566 144/1688 [=>............................] - ETA: 3s - loss: 0.1379 - accuracy: 0.9562 168/1688 [=>............................] - ETA: 3s - loss: 0.1399 - accuracy: 0.9560 192/1688 [==>...........................] - ETA: 3s - loss: 0.1415 - accuracy: 0.9557 216/1688 [==>...........................] - ETA: 3s - loss: 0.1430 - accuracy: 0.9554 240/1688 [===>..........................] - ETA: 3s - loss: 0.1439 - accuracy: 0.9554 265/1688 [===>..........................] - ETA: 3s - loss: 0.1445 - accuracy: 0.9554 289/1688 [====>.........................] - ETA: 2s - loss: 0.1448 - accuracy: 0.9555 314/1688 [====>.........................] - ETA: 2s - loss: 0.1450 - accuracy: 0.9556 337/1688 [====>.........................] - ETA: 2s - loss: 0.1450 - accuracy: 0.9557 361/1688 [=====>........................] - ETA: 2s - loss: 0.1450 - accuracy: 0.9559 385/1688 [=====>........................] - ETA: 2s - loss: 0.1448 - accuracy: 0.9560 409/1688 [======>.......................] - ETA: 2s - loss: 0.1444 - accuracy: 0.9562 434/1688 [======>.......................] - ETA: 2s - loss: 0.1442 - accuracy: 0.9564 458/1688 [=======>......................] - ETA: 2s - loss: 0.1441 - accuracy: 0.9565 482/1688 [=======>......................] - ETA: 2s - loss: 0.1440 - accuracy: 0.9566 506/1688 [=======>......................] - ETA: 2s - loss: 0.1438 - accuracy: 0.9567 530/1688 [========>.....................] - ETA: 2s - loss: 0.1437 - accuracy: 0.9569 554/1688 [========>.....................] - ETA: 2s - loss: 0.1434 - accuracy: 0.9570 578/1688 [=========>....................] - ETA: 2s - loss: 0.1432 - accuracy: 0.9571 602/1688 [=========>....................] - ETA: 2s - loss: 0.1430 - accuracy: 0.9572 626/1688 [==========>...................] - ETA: 2s - loss: 0.1428 - accuracy: 0.9574 649/1688 [==========>...................] - ETA: 2s - loss: 0.1426 - accuracy: 0.9575 673/1688 [==========>...................] - ETA: 2s - loss: 0.1424 - accuracy: 0.9576 697/1688 [===========>..................] - ETA: 2s - loss: 0.1422 - accuracy: 0.9577 722/1688 [===========>..................] - ETA: 2s - loss: 0.1421 - accuracy: 0.9578 746/1688 [============>.................] - ETA: 1s - loss: 0.1419 - accuracy: 0.9578 770/1688 [============>.................] - ETA: 1s - loss: 0.1418 - accuracy: 0.9579 794/1688 [=============>................] - ETA: 1s - loss: 0.1416 - accuracy: 0.9580 819/1688 [=============>................] - ETA: 1s - loss: 0.1415 - accuracy: 0.9581 843/1688 [=============>................] - ETA: 1s - loss: 0.1413 - accuracy: 0.9581 866/1688 [==============>...............] - ETA: 1s - loss: 0.1412 - accuracy: 0.9582 890/1688 [==============>...............] - ETA: 1s - loss: 0.1410 - accuracy: 0.9583 915/1688 [===============>..............] - ETA: 1s - loss: 0.1408 - accuracy: 0.9583 939/1688 [===============>..............] - ETA: 1s - loss: 0.1406 - accuracy: 0.9584 964/1688 [================>.............] - ETA: 1s - loss: 0.1405 - accuracy: 0.9585 989/1688 [================>.............] - ETA: 1s - loss: 0.1403 - accuracy: 0.9585 1012/1688 [================>.............] - ETA: 1s - loss: 0.1401 - accuracy: 0.9586 1035/1688 [=================>............] - ETA: 1s - loss: 0.1399 - accuracy: 0.9587 1059/1688 [=================>............] - ETA: 1s - loss: 0.1398 - accuracy: 0.9587 1083/1688 [==================>...........] - ETA: 1s - loss: 0.1396 - accuracy: 0.9588 1107/1688 [==================>...........] - ETA: 1s - loss: 0.1394 - accuracy: 0.9588 1131/1688 [===================>..........] - ETA: 1s - loss: 0.1392 - accuracy: 0.9589 1155/1688 [===================>..........] - ETA: 1s - loss: 0.1390 - accuracy: 0.9590 1178/1688 [===================>..........] - ETA: 1s - loss: 0.1388 - accuracy: 0.9590 1201/1688 [====================>.........] - ETA: 1s - loss: 0.1386 - accuracy: 0.9591 1226/1688 [====================>.........] - ETA: 0s - loss: 0.1383 - accuracy: 0.9592 1251/1688 [=====================>........] - ETA: 0s - loss: 0.1381 - accuracy: 0.9592 1275/1688 [=====================>........] - ETA: 0s - loss: 0.1379 - accuracy: 0.9593 1299/1688 [======================>.......] - ETA: 0s - loss: 0.1377 - accuracy: 0.9594 1323/1688 [======================>.......] - ETA: 0s - loss: 0.1375 - accuracy: 0.9594 1346/1688 [======================>.......] - ETA: 0s - loss: 0.1373 - accuracy: 0.9595 1370/1688 [=======================>......] - ETA: 0s - loss: 0.1371 - accuracy: 0.9596 1394/1688 [=======================>......] - ETA: 0s - loss: 0.1369 - accuracy: 0.9596 1419/1688 [========================>.....] - ETA: 0s - loss: 0.1367 - accuracy: 0.9597 1443/1688 [========================>.....] - ETA: 0s - loss: 0.1365 - accuracy: 0.9598 1467/1688 [=========================>....] - ETA: 0s - loss: 0.1363 - accuracy: 0.9598 1491/1688 [=========================>....] - ETA: 0s - loss: 0.1361 - accuracy: 0.9599 1515/1688 [=========================>....] - ETA: 0s - loss: 0.1359 - accuracy: 0.9599 1539/1688 [==========================>...] - ETA: 0s - loss: 0.1358 - accuracy: 0.9600 1564/1688 [==========================>...] - ETA: 0s - loss: 0.1356 - accuracy: 0.9601 1589/1688 [===========================>..] - ETA: 0s - loss: 0.1354 - accuracy: 0.9601 1613/1688 [===========================>..] - ETA: 0s - loss: 0.1352 - accuracy: 0.9602 1638/1688 [============================>.] - ETA: 0s - loss: 0.1350 - accuracy: 0.9602 1663/1688 [============================>.] - ETA: 0s - loss: 0.1349 - accuracy: 0.9603 1688/1688 [==============================] - ETA: 0s - loss: 0.1347 - accuracy: 0.9603 1688/1688 [==============================] - 4s 2ms/step - loss: 0.1347 - accuracy: 0.9603 - val_loss: 0.0868 - val_accuracy: 0.9773 Epoch 3/10 1/1688 [..............................] - ETA: 4s - loss: 0.1929 - accuracy: 0.9375 24/1688 [..............................] - ETA: 3s - loss: 0.1050 - accuracy: 0.9680 47/1688 [..............................] - ETA: 3s - loss: 0.0979 - accuracy: 0.9716 71/1688 [>.............................] - ETA: 3s - loss: 0.0954 - accuracy: 0.9729 96/1688 [>.............................] - ETA: 3s - loss: 0.0939 - accuracy: 0.9736 120/1688 [=>............................] - ETA: 3s - loss: 0.0926 - accuracy: 0.9741 144/1688 [=>............................] - ETA: 3s - loss: 0.0919 - accuracy: 0.9743 168/1688 [=>............................] - ETA: 3s - loss: 0.0918 - accuracy: 0.9743 192/1688 [==>...........................] - ETA: 3s - loss: 0.0918 - accuracy: 0.9742 217/1688 [==>...........................] - ETA: 3s - loss: 0.0918 - accuracy: 0.9742 242/1688 [===>..........................] - ETA: 3s - loss: 0.0917 - accuracy: 0.9742 267/1688 [===>..........................] - ETA: 2s - loss: 0.0919 - accuracy: 0.9741 291/1688 [====>.........................] - ETA: 2s - loss: 0.0923 - accuracy: 0.9740 315/1688 [====>.........................] - ETA: 2s - loss: 0.0924 - accuracy: 0.9739 339/1688 [=====>........................] - ETA: 2s - loss: 0.0926 - accuracy: 0.9737 363/1688 [=====>........................] - ETA: 2s - loss: 0.0927 - accuracy: 0.9736 387/1688 [=====>........................] - ETA: 2s - loss: 0.0929 - accuracy: 0.9735 411/1688 [======>.......................] - ETA: 2s - loss: 0.0931 - accuracy: 0.9734 436/1688 [======>.......................] - ETA: 2s - loss: 0.0934 - accuracy: 0.9733 461/1688 [=======>......................] - ETA: 2s - loss: 0.0936 - accuracy: 0.9732 486/1688 [=======>......................] - ETA: 2s - loss: 0.0938 - accuracy: 0.9731 510/1688 [========>.....................] - ETA: 2s - loss: 0.0939 - accuracy: 0.9730 534/1688 [========>.....................] - ETA: 2s - loss: 0.0940 - accuracy: 0.9730 558/1688 [========>.....................] - ETA: 2s - loss: 0.0941 - accuracy: 0.9729 581/1688 [=========>....................] - ETA: 2s - loss: 0.0942 - accuracy: 0.9729 605/1688 [=========>....................] - ETA: 2s - loss: 0.0943 - accuracy: 0.9728 628/1688 [==========>...................] - ETA: 2s - loss: 0.0944 - accuracy: 0.9728 651/1688 [==========>...................] - ETA: 2s - loss: 0.0945 - accuracy: 0.9727 674/1688 [==========>...................] - ETA: 2s - loss: 0.0945 - accuracy: 0.9727 697/1688 [===========>..................] - ETA: 2s - loss: 0.0946 - accuracy: 0.9726 721/1688 [===========>..................] - ETA: 2s - loss: 0.0947 - accuracy: 0.9726 746/1688 [============>.................] - ETA: 1s - loss: 0.0947 - accuracy: 0.9726 770/1688 [============>.................] - ETA: 1s - loss: 0.0948 - accuracy: 0.9725 794/1688 [=============>................] - ETA: 1s - loss: 0.0948 - accuracy: 0.9725 817/1688 [=============>................] - ETA: 1s - loss: 0.0948 - accuracy: 0.9725 840/1688 [=============>................] - ETA: 1s - loss: 0.0948 - accuracy: 0.9725 863/1688 [==============>...............] - ETA: 1s - loss: 0.0947 - accuracy: 0.9725 886/1688 [==============>...............] - ETA: 1s - loss: 0.0947 - accuracy: 0.9724 910/1688 [===============>..............] - ETA: 1s - loss: 0.0947 - accuracy: 0.9724 934/1688 [===============>..............] - ETA: 1s - loss: 0.0947 - accuracy: 0.9724 957/1688 [================>.............] - ETA: 1s - loss: 0.0947 - accuracy: 0.9724 980/1688 [================>.............] - ETA: 1s - loss: 0.0947 - accuracy: 0.9724 1004/1688 [================>.............] - ETA: 1s - loss: 0.0947 - accuracy: 0.9724 1027/1688 [=================>............] - ETA: 1s - loss: 0.0947 - accuracy: 0.9724 1051/1688 [=================>............] - ETA: 1s - loss: 0.0947 - accuracy: 0.9724 1075/1688 [==================>...........] - ETA: 1s - loss: 0.0947 - accuracy: 0.9724 1099/1688 [==================>...........] - ETA: 1s - loss: 0.0946 - accuracy: 0.9724 1124/1688 [==================>...........] - ETA: 1s - loss: 0.0946 - accuracy: 0.9724 1149/1688 [===================>..........] - ETA: 1s - loss: 0.0946 - accuracy: 0.9724 1174/1688 [===================>..........] - ETA: 1s - loss: 0.0945 - accuracy: 0.9724 1197/1688 [====================>.........] - ETA: 1s - loss: 0.0945 - accuracy: 0.9724 1221/1688 [====================>.........] - ETA: 0s - loss: 0.0945 - accuracy: 0.9724 1245/1688 [=====================>........] - ETA: 0s - loss: 0.0945 - accuracy: 0.9724 1269/1688 [=====================>........] - ETA: 0s - loss: 0.0945 - accuracy: 0.9724 1293/1688 [=====================>........] - ETA: 0s - loss: 0.0944 - accuracy: 0.9724 1316/1688 [======================>.......] - ETA: 0s - loss: 0.0944 - accuracy: 0.9724 1340/1688 [======================>.......] - ETA: 0s - loss: 0.0944 - accuracy: 0.9724 1363/1688 [=======================>......] - ETA: 0s - loss: 0.0943 - accuracy: 0.9724 1387/1688 [=======================>......] - ETA: 0s - loss: 0.0943 - accuracy: 0.9724 1411/1688 [========================>.....] - ETA: 0s - loss: 0.0942 - accuracy: 0.9724 1435/1688 [========================>.....] - ETA: 0s - loss: 0.0942 - accuracy: 0.9724 1459/1688 [========================>.....] - ETA: 0s - loss: 0.0941 - accuracy: 0.9724 1483/1688 [=========================>....] - ETA: 0s - loss: 0.0941 - accuracy: 0.9725 1507/1688 [=========================>....] - ETA: 0s - loss: 0.0940 - accuracy: 0.9725 1532/1688 [==========================>...] - ETA: 0s - loss: 0.0939 - accuracy: 0.9725 1556/1688 [==========================>...] - ETA: 0s - loss: 0.0939 - accuracy: 0.9725 1580/1688 [===========================>..] - ETA: 0s - loss: 0.0938 - accuracy: 0.9725 1604/1688 [===========================>..] - ETA: 0s - loss: 0.0937 - accuracy: 0.9725 1628/1688 [===========================>..] - ETA: 0s - loss: 0.0937 - accuracy: 0.9726 1652/1688 [============================>.] - ETA: 0s - loss: 0.0936 - accuracy: 0.9726 1676/1688 [============================>.] - ETA: 0s - loss: 0.0935 - accuracy: 0.9726 1688/1688 [==============================] - 4s 2ms/step - loss: 0.0935 - accuracy: 0.9726 - val_loss: 0.0695 - val_accuracy: 0.9822 Epoch 4/10 1/1688 [..............................] - ETA: 4s - loss: 0.0160 - accuracy: 1.0000 24/1688 [..............................] - ETA: 3s - loss: 0.0404 - accuracy: 0.9840 48/1688 [..............................] - ETA: 3s - loss: 0.0448 - accuracy: 0.9836 72/1688 [>.............................] - ETA: 3s - loss: 0.0485 - accuracy: 0.9836 96/1688 [>.............................] - ETA: 3s - loss: 0.0519 - accuracy: 0.9833 120/1688 [=>............................] - ETA: 3s - loss: 0.0557 - accuracy: 0.9826 144/1688 [=>............................] - ETA: 3s - loss: 0.0585 - accuracy: 0.9820 168/1688 [=>............................] - ETA: 3s - loss: 0.0606 - accuracy: 0.9816 191/1688 [==>...........................] - ETA: 3s - loss: 0.0624 - accuracy: 0.9813 215/1688 [==>...........................] - ETA: 3s - loss: 0.0639 - accuracy: 0.9810 239/1688 [===>..........................] - ETA: 3s - loss: 0.0652 - accuracy: 0.9807 264/1688 [===>..........................] - ETA: 3s - loss: 0.0662 - accuracy: 0.9805 289/1688 [====>.........................] - ETA: 2s - loss: 0.0670 - accuracy: 0.9804 313/1688 [====>.........................] - ETA: 2s - loss: 0.0677 - accuracy: 0.9803 337/1688 [====>.........................] - ETA: 2s - loss: 0.0681 - accuracy: 0.9802 361/1688 [=====>........................] - ETA: 2s - loss: 0.0684 - accuracy: 0.9802 385/1688 [=====>........................] - ETA: 2s - loss: 0.0687 - accuracy: 0.9802 410/1688 [======>.......................] - ETA: 2s - loss: 0.0691 - accuracy: 0.9801 434/1688 [======>.......................] - ETA: 2s - loss: 0.0693 - accuracy: 0.9801 458/1688 [=======>......................] - ETA: 2s - loss: 0.0696 - accuracy: 0.9800 481/1688 [=======>......................] - ETA: 2s - loss: 0.0698 - accuracy: 0.9800 505/1688 [=======>......................] - ETA: 2s - loss: 0.0699 - accuracy: 0.9800 528/1688 [========>.....................] - ETA: 2s - loss: 0.0700 - accuracy: 0.9800 552/1688 [========>.....................] - ETA: 2s - loss: 0.0701 - accuracy: 0.9800 577/1688 [=========>....................] - ETA: 2s - loss: 0.0701 - accuracy: 0.9800 602/1688 [=========>....................] - ETA: 2s - loss: 0.0702 - accuracy: 0.9800 626/1688 [==========>...................] - ETA: 2s - loss: 0.0702 - accuracy: 0.9800 650/1688 [==========>...................] - ETA: 2s - loss: 0.0703 - accuracy: 0.9800 674/1688 [==========>...................] - ETA: 2s - loss: 0.0703 - accuracy: 0.9801 698/1688 [===========>..................] - ETA: 2s - loss: 0.0704 - accuracy: 0.9801 723/1688 [===========>..................] - ETA: 2s - loss: 0.0705 - accuracy: 0.9801 748/1688 [============>.................] - ETA: 1s - loss: 0.0705 - accuracy: 0.9801 774/1688 [============>.................] - ETA: 1s - loss: 0.0705 - accuracy: 0.9801 799/1688 [=============>................] - ETA: 1s - loss: 0.0705 - accuracy: 0.9801 823/1688 [=============>................] - ETA: 1s - loss: 0.0706 - accuracy: 0.9800 847/1688 [==============>...............] - ETA: 1s - loss: 0.0706 - accuracy: 0.9800 871/1688 [==============>...............] - ETA: 1s - loss: 0.0705 - accuracy: 0.9800 896/1688 [==============>...............] - ETA: 1s - loss: 0.0705 - accuracy: 0.9800 921/1688 [===============>..............] - ETA: 1s - loss: 0.0705 - accuracy: 0.9800 945/1688 [===============>..............] - ETA: 1s - loss: 0.0705 - accuracy: 0.9800 969/1688 [================>.............] - ETA: 1s - loss: 0.0705 - accuracy: 0.9801 993/1688 [================>.............] - ETA: 1s - loss: 0.0705 - accuracy: 0.9800 1018/1688 [=================>............] - ETA: 1s - loss: 0.0705 - accuracy: 0.9800 1042/1688 [=================>............] - ETA: 1s - loss: 0.0705 - accuracy: 0.9800 1066/1688 [=================>............] - ETA: 1s - loss: 0.0705 - accuracy: 0.9800 1091/1688 [==================>...........] - ETA: 1s - loss: 0.0705 - accuracy: 0.9800 1117/1688 [==================>...........] - ETA: 1s - loss: 0.0705 - accuracy: 0.9800 1142/1688 [===================>..........] - ETA: 1s - loss: 0.0705 - accuracy: 0.9800 1167/1688 [===================>..........] - ETA: 1s - loss: 0.0705 - accuracy: 0.9800 1191/1688 [====================>.........] - ETA: 1s - loss: 0.0705 - accuracy: 0.9800 1215/1688 [====================>.........] - ETA: 0s - loss: 0.0705 - accuracy: 0.9800 1240/1688 [=====================>........] - ETA: 0s - loss: 0.0705 - accuracy: 0.9800 1264/1688 [=====================>........] - ETA: 0s - loss: 0.0705 - accuracy: 0.9800 1288/1688 [=====================>........] - ETA: 0s - loss: 0.0705 - accuracy: 0.9800 1313/1688 [======================>.......] - ETA: 0s - loss: 0.0705 - accuracy: 0.9800 1337/1688 [======================>.......] - ETA: 0s - loss: 0.0705 - accuracy: 0.9800 1362/1688 [=======================>......] - ETA: 0s - loss: 0.0705 - accuracy: 0.9800 1386/1688 [=======================>......] - ETA: 0s - loss: 0.0705 - accuracy: 0.9799 1410/1688 [========================>.....] - ETA: 0s - loss: 0.0706 - accuracy: 0.9799 1435/1688 [========================>.....] - ETA: 0s - loss: 0.0706 - accuracy: 0.9799 1461/1688 [========================>.....] - ETA: 0s - loss: 0.0706 - accuracy: 0.9799 1485/1688 [=========================>....] - ETA: 0s - loss: 0.0706 - accuracy: 0.9799 1510/1688 [=========================>....] - ETA: 0s - loss: 0.0707 - accuracy: 0.9798 1535/1688 [==========================>...] - ETA: 0s - loss: 0.0707 - accuracy: 0.9798 1560/1688 [==========================>...] - ETA: 0s - loss: 0.0707 - accuracy: 0.9798 1584/1688 [===========================>..] - ETA: 0s - loss: 0.0707 - accuracy: 0.9798 1608/1688 [===========================>..] - ETA: 0s - loss: 0.0708 - accuracy: 0.9798 1632/1688 [============================>.] - ETA: 0s - loss: 0.0708 - accuracy: 0.9797 1657/1688 [============================>.] - ETA: 0s - loss: 0.0708 - accuracy: 0.9797 1682/1688 [============================>.] - ETA: 0s - loss: 0.0708 - accuracy: 0.9797 1688/1688 [==============================] - 4s 2ms/step - loss: 0.0708 - accuracy: 0.9797 - val_loss: 0.0681 - val_accuracy: 0.9817 Epoch 5/10 1/1688 [..............................] - ETA: 4s - loss: 0.0042 - accuracy: 1.0000 25/1688 [..............................] - ETA: 3s - loss: 0.0314 - accuracy: 0.9931 49/1688 [..............................] - ETA: 3s - loss: 0.0427 - accuracy: 0.9886 74/1688 [>.............................] - ETA: 3s - loss: 0.0491 - accuracy: 0.9859 98/1688 [>.............................] - ETA: 3s - loss: 0.0533 - accuracy: 0.9842 122/1688 [=>............................] - ETA: 3s - loss: 0.0562 - accuracy: 0.9832 147/1688 [=>............................] - ETA: 3s - loss: 0.0577 - accuracy: 0.9828 172/1688 [==>...........................] - ETA: 3s - loss: 0.0580 - accuracy: 0.9827 197/1688 [==>...........................] - ETA: 3s - loss: 0.0580 - accuracy: 0.9827 221/1688 [==>...........................] - ETA: 3s - loss: 0.0582 - accuracy: 0.9826 245/1688 [===>..........................] - ETA: 3s - loss: 0.0584 - accuracy: 0.9826 270/1688 [===>..........................] - ETA: 2s - loss: 0.0583 - accuracy: 0.9826 295/1688 [====>.........................] - ETA: 2s - loss: 0.0583 - accuracy: 0.9826 320/1688 [====>.........................] - ETA: 2s - loss: 0.0584 - accuracy: 0.9826 344/1688 [=====>........................] - ETA: 2s - loss: 0.0584 - accuracy: 0.9825 367/1688 [=====>........................] - ETA: 2s - loss: 0.0585 - accuracy: 0.9825 391/1688 [=====>........................] - ETA: 2s - loss: 0.0586 - accuracy: 0.9824 416/1688 [======>.......................] - ETA: 2s - loss: 0.0586 - accuracy: 0.9824 441/1688 [======>.......................] - ETA: 2s - loss: 0.0586 - accuracy: 0.9824 466/1688 [=======>......................] - ETA: 2s - loss: 0.0588 - accuracy: 0.9823 491/1688 [=======>......................] - ETA: 2s - loss: 0.0589 - accuracy: 0.9822 515/1688 [========>.....................] - ETA: 2s - loss: 0.0591 - accuracy: 0.9822 540/1688 [========>.....................] - ETA: 2s - loss: 0.0592 - accuracy: 0.9822 565/1688 [=========>....................] - ETA: 2s - loss: 0.0593 - accuracy: 0.9821 589/1688 [=========>....................] - ETA: 2s - loss: 0.0594 - accuracy: 0.9821 613/1688 [=========>....................] - ETA: 2s - loss: 0.0595 - accuracy: 0.9820 637/1688 [==========>...................] - ETA: 2s - loss: 0.0596 - accuracy: 0.9820 661/1688 [==========>...................] - ETA: 2s - loss: 0.0597 - accuracy: 0.9819 685/1688 [===========>..................] - ETA: 2s - loss: 0.0598 - accuracy: 0.9819 710/1688 [===========>..................] - ETA: 2s - loss: 0.0599 - accuracy: 0.9818 735/1688 [============>.................] - ETA: 1s - loss: 0.0600 - accuracy: 0.9818 760/1688 [============>.................] - ETA: 1s - loss: 0.0602 - accuracy: 0.9817 784/1688 [============>.................] - ETA: 1s - loss: 0.0603 - accuracy: 0.9817 808/1688 [=============>................] - ETA: 1s - loss: 0.0604 - accuracy: 0.9817 833/1688 [=============>................] - ETA: 1s - loss: 0.0605 - accuracy: 0.9816 858/1688 [==============>...............] - ETA: 1s - loss: 0.0606 - accuracy: 0.9816 882/1688 [==============>...............] - ETA: 1s - loss: 0.0606 - accuracy: 0.9816 905/1688 [===============>..............] - ETA: 1s - loss: 0.0607 - accuracy: 0.9815 930/1688 [===============>..............] - ETA: 1s - loss: 0.0608 - accuracy: 0.9815 954/1688 [===============>..............] - ETA: 1s - loss: 0.0608 - accuracy: 0.9815 979/1688 [================>.............] - ETA: 1s - loss: 0.0609 - accuracy: 0.9815 1003/1688 [================>.............] - ETA: 1s - loss: 0.0610 - accuracy: 0.9815 1027/1688 [=================>............] - ETA: 1s - loss: 0.0610 - accuracy: 0.9815 1051/1688 [=================>............] - ETA: 1s - loss: 0.0611 - accuracy: 0.9815 1075/1688 [==================>...........] - ETA: 1s - loss: 0.0611 - accuracy: 0.9814 1099/1688 [==================>...........] - ETA: 1s - loss: 0.0611 - accuracy: 0.9814 1123/1688 [==================>...........] - ETA: 1s - loss: 0.0612 - accuracy: 0.9814 1147/1688 [===================>..........] - ETA: 1s - loss: 0.0612 - accuracy: 0.9814 1172/1688 [===================>..........] - ETA: 1s - loss: 0.0612 - accuracy: 0.9814 1196/1688 [====================>.........] - ETA: 1s - loss: 0.0612 - accuracy: 0.9814 1221/1688 [====================>.........] - ETA: 0s - loss: 0.0613 - accuracy: 0.9814 1244/1688 [=====================>........] - ETA: 0s - loss: 0.0613 - accuracy: 0.9814 1269/1688 [=====================>........] - ETA: 0s - loss: 0.0613 - accuracy: 0.9814 1294/1688 [=====================>........] - ETA: 0s - loss: 0.0613 - accuracy: 0.9814 1318/1688 [======================>.......] - ETA: 0s - loss: 0.0614 - accuracy: 0.9814 1343/1688 [======================>.......] - ETA: 0s - loss: 0.0614 - accuracy: 0.9814 1368/1688 [=======================>......] - ETA: 0s - loss: 0.0614 - accuracy: 0.9813 1391/1688 [=======================>......] - ETA: 0s - loss: 0.0614 - accuracy: 0.9813 1414/1688 [========================>.....] - ETA: 0s - loss: 0.0615 - accuracy: 0.9813 1437/1688 [========================>.....] - ETA: 0s - loss: 0.0615 - accuracy: 0.9813 1461/1688 [========================>.....] - ETA: 0s - loss: 0.0615 - accuracy: 0.9813 1486/1688 [=========================>....] - ETA: 0s - loss: 0.0615 - accuracy: 0.9813 1511/1688 [=========================>....] - ETA: 0s - loss: 0.0615 - accuracy: 0.9813 1535/1688 [==========================>...] - ETA: 0s - loss: 0.0615 - accuracy: 0.9813 1559/1688 [==========================>...] - ETA: 0s - loss: 0.0615 - accuracy: 0.9813 1583/1688 [===========================>..] - ETA: 0s - loss: 0.0616 - accuracy: 0.9813 1607/1688 [===========================>..] - ETA: 0s - loss: 0.0616 - accuracy: 0.9813 1631/1688 [===========================>..] - ETA: 0s - loss: 0.0616 - accuracy: 0.9813 1655/1688 [============================>.] - ETA: 0s - loss: 0.0616 - accuracy: 0.9813 1679/1688 [============================>.] - ETA: 0s - loss: 0.0616 - accuracy: 0.9813 1688/1688 [==============================] - 4s 2ms/step - loss: 0.0616 - accuracy: 0.9813 - val_loss: 0.0611 - val_accuracy: 0.9835 Epoch 6/10 1/1688 [..............................] - ETA: 4s - loss: 0.0868 - accuracy: 0.9375 26/1688 [..............................] - ETA: 3s - loss: 0.0590 - accuracy: 0.9779 51/1688 [..............................] - ETA: 3s - loss: 0.0524 - accuracy: 0.9826 74/1688 [>.............................] - ETA: 3s - loss: 0.0494 - accuracy: 0.9844 98/1688 [>.............................] - ETA: 3s - loss: 0.0476 - accuracy: 0.9855 123/1688 [=>............................] - ETA: 3s - loss: 0.0466 - accuracy: 0.9861 148/1688 [=>............................] - ETA: 3s - loss: 0.0466 - accuracy: 0.9863 173/1688 [==>...........................] - ETA: 3s - loss: 0.0467 - accuracy: 0.9864 197/1688 [==>...........................] - ETA: 3s - loss: 0.0471 - accuracy: 0.9864 222/1688 [==>...........................] - ETA: 3s - loss: 0.0474 - accuracy: 0.9863 246/1688 [===>..........................] - ETA: 3s - loss: 0.0476 - accuracy: 0.9863 270/1688 [===>..........................] - ETA: 2s - loss: 0.0478 - accuracy: 0.9862 294/1688 [====>.........................] - ETA: 2s - loss: 0.0479 - accuracy: 0.9861 319/1688 [====>.........................] - ETA: 2s - loss: 0.0481 - accuracy: 0.9860 344/1688 [=====>........................] - ETA: 2s - loss: 0.0483 - accuracy: 0.9860 369/1688 [=====>........................] - ETA: 2s - loss: 0.0485 - accuracy: 0.9859 393/1688 [=====>........................] - ETA: 2s - loss: 0.0487 - accuracy: 0.9858 417/1688 [======>.......................] - ETA: 2s - loss: 0.0488 - accuracy: 0.9858 440/1688 [======>.......................] - ETA: 2s - loss: 0.0489 - accuracy: 0.9858 464/1688 [=======>......................] - ETA: 2s - loss: 0.0489 - accuracy: 0.9857 489/1688 [=======>......................] - ETA: 2s - loss: 0.0490 - accuracy: 0.9857 513/1688 [========>.....................] - ETA: 2s - loss: 0.0490 - accuracy: 0.9857 537/1688 [========>.....................] - ETA: 2s - loss: 0.0490 - accuracy: 0.9857 562/1688 [========>.....................] - ETA: 2s - loss: 0.0491 - accuracy: 0.9857 586/1688 [=========>....................] - ETA: 2s - loss: 0.0491 - accuracy: 0.9857 611/1688 [=========>....................] - ETA: 2s - loss: 0.0492 - accuracy: 0.9857 635/1688 [==========>...................] - ETA: 2s - loss: 0.0492 - accuracy: 0.9857 660/1688 [==========>...................] - ETA: 2s - loss: 0.0493 - accuracy: 0.9856 684/1688 [===========>..................] - ETA: 2s - loss: 0.0493 - accuracy: 0.9856 708/1688 [===========>..................] - ETA: 2s - loss: 0.0494 - accuracy: 0.9856 732/1688 [============>.................] - ETA: 2s - loss: 0.0495 - accuracy: 0.9856 756/1688 [============>.................] - ETA: 1s - loss: 0.0496 - accuracy: 0.9856 780/1688 [============>.................] - ETA: 1s - loss: 0.0497 - accuracy: 0.9855 805/1688 [=============>................] - ETA: 1s - loss: 0.0498 - accuracy: 0.9855 830/1688 [=============>................] - ETA: 1s - loss: 0.0499 - accuracy: 0.9855 855/1688 [==============>...............] - ETA: 1s - loss: 0.0499 - accuracy: 0.9855 879/1688 [==============>...............] - ETA: 1s - loss: 0.0500 - accuracy: 0.9854 903/1688 [===============>..............] - ETA: 1s - loss: 0.0501 - accuracy: 0.9854 927/1688 [===============>..............] - ETA: 1s - loss: 0.0502 - accuracy: 0.9854 951/1688 [===============>..............] - ETA: 1s - loss: 0.0503 - accuracy: 0.9854 975/1688 [================>.............] - ETA: 1s - loss: 0.0503 - accuracy: 0.9853 1000/1688 [================>.............] - ETA: 1s - loss: 0.0504 - accuracy: 0.9853 1024/1688 [=================>............] - ETA: 1s - loss: 0.0505 - accuracy: 0.9853 1048/1688 [=================>............] - ETA: 1s - loss: 0.0505 - accuracy: 0.9853 1073/1688 [==================>...........] - ETA: 1s - loss: 0.0506 - accuracy: 0.9853 1098/1688 [==================>...........] - ETA: 1s - loss: 0.0506 - accuracy: 0.9852 1121/1688 [==================>...........] - ETA: 1s - loss: 0.0507 - accuracy: 0.9852 1145/1688 [===================>..........] - ETA: 1s - loss: 0.0507 - accuracy: 0.9852 1169/1688 [===================>..........] - ETA: 1s - loss: 0.0507 - accuracy: 0.9852 1193/1688 [====================>.........] - ETA: 1s - loss: 0.0508 - accuracy: 0.9852 1217/1688 [====================>.........] - ETA: 0s - loss: 0.0508 - accuracy: 0.9852 1241/1688 [=====================>........] - ETA: 0s - loss: 0.0509 - accuracy: 0.9851 1265/1688 [=====================>........] - ETA: 0s - loss: 0.0509 - accuracy: 0.9851 1289/1688 [=====================>........] - ETA: 0s - loss: 0.0509 - accuracy: 0.9851 1314/1688 [======================>.......] - ETA: 0s - loss: 0.0510 - accuracy: 0.9851 1339/1688 [======================>.......] - ETA: 0s - loss: 0.0510 - accuracy: 0.9851 1364/1688 [=======================>......] - ETA: 0s - loss: 0.0510 - accuracy: 0.9851 1388/1688 [=======================>......] - ETA: 0s - loss: 0.0510 - accuracy: 0.9850 1413/1688 [========================>.....] - ETA: 0s - loss: 0.0511 - accuracy: 0.9850 1437/1688 [========================>.....] - ETA: 0s - loss: 0.0511 - accuracy: 0.9850 1461/1688 [========================>.....] - ETA: 0s - loss: 0.0511 - accuracy: 0.9850 1485/1688 [=========================>....] - ETA: 0s - loss: 0.0512 - accuracy: 0.9850 1509/1688 [=========================>....] - ETA: 0s - loss: 0.0512 - accuracy: 0.9850 1534/1688 [==========================>...] - ETA: 0s - loss: 0.0513 - accuracy: 0.9850 1559/1688 [==========================>...] - ETA: 0s - loss: 0.0513 - accuracy: 0.9849 1583/1688 [===========================>..] - ETA: 0s - loss: 0.0513 - accuracy: 0.9849 1606/1688 [===========================>..] - ETA: 0s - loss: 0.0514 - accuracy: 0.9849 1629/1688 [===========================>..] - ETA: 0s - loss: 0.0514 - accuracy: 0.9849 1654/1688 [============================>.] - ETA: 0s - loss: 0.0515 - accuracy: 0.9849 1679/1688 [============================>.] - ETA: 0s - loss: 0.0515 - accuracy: 0.9849 1688/1688 [==============================] - 4s 2ms/step - loss: 0.0515 - accuracy: 0.9849 - val_loss: 0.0609 - val_accuracy: 0.9838 Epoch 7/10 1/1688 [..............................] - ETA: 3s - loss: 0.0144 - accuracy: 1.0000 25/1688 [..............................] - ETA: 3s - loss: 0.0484 - accuracy: 0.9874 50/1688 [..............................] - ETA: 3s - loss: 0.0540 - accuracy: 0.9845 75/1688 [>.............................] - ETA: 3s - loss: 0.0549 - accuracy: 0.9837 99/1688 [>.............................] - ETA: 3s - loss: 0.0551 - accuracy: 0.9836 122/1688 [=>............................] - ETA: 3s - loss: 0.0543 - accuracy: 0.9840 146/1688 [=>............................] - ETA: 3s - loss: 0.0537 - accuracy: 0.9842 171/1688 [==>...........................] - ETA: 3s - loss: 0.0533 - accuracy: 0.9845 196/1688 [==>...........................] - ETA: 3s - loss: 0.0531 - accuracy: 0.9846 220/1688 [==>...........................] - ETA: 3s - loss: 0.0529 - accuracy: 0.9847 245/1688 [===>..........................] - ETA: 3s - loss: 0.0528 - accuracy: 0.9848 269/1688 [===>..........................] - ETA: 2s - loss: 0.0526 - accuracy: 0.9849 294/1688 [====>.........................] - ETA: 2s - loss: 0.0525 - accuracy: 0.9850 319/1688 [====>.........................] - ETA: 2s - loss: 0.0523 - accuracy: 0.9851 344/1688 [=====>........................] - ETA: 2s - loss: 0.0521 - accuracy: 0.9851 369/1688 [=====>........................] - ETA: 2s - loss: 0.0519 - accuracy: 0.9852 393/1688 [=====>........................] - ETA: 2s - loss: 0.0517 - accuracy: 0.9853 417/1688 [======>.......................] - ETA: 2s - loss: 0.0515 - accuracy: 0.9853 440/1688 [======>.......................] - ETA: 2s - loss: 0.0514 - accuracy: 0.9853 464/1688 [=======>......................] - ETA: 2s - loss: 0.0512 - accuracy: 0.9853 488/1688 [=======>......................] - ETA: 2s - loss: 0.0511 - accuracy: 0.9853 513/1688 [========>.....................] - ETA: 2s - loss: 0.0510 - accuracy: 0.9853 537/1688 [========>.....................] - ETA: 2s - loss: 0.0510 - accuracy: 0.9853 562/1688 [========>.....................] - ETA: 2s - loss: 0.0509 - accuracy: 0.9853 587/1688 [=========>....................] - ETA: 2s - loss: 0.0508 - accuracy: 0.9853 610/1688 [=========>....................] - ETA: 2s - loss: 0.0507 - accuracy: 0.9853 634/1688 [==========>...................] - ETA: 2s - loss: 0.0506 - accuracy: 0.9854 659/1688 [==========>...................] - ETA: 2s - loss: 0.0505 - accuracy: 0.9854 683/1688 [===========>..................] - ETA: 2s - loss: 0.0504 - accuracy: 0.9854 707/1688 [===========>..................] - ETA: 2s - loss: 0.0503 - accuracy: 0.9854 731/1688 [===========>..................] - ETA: 2s - loss: 0.0502 - accuracy: 0.9854 756/1688 [============>.................] - ETA: 1s - loss: 0.0501 - accuracy: 0.9854 779/1688 [============>.................] - ETA: 1s - loss: 0.0500 - accuracy: 0.9854 803/1688 [=============>................] - ETA: 1s - loss: 0.0499 - accuracy: 0.9855 827/1688 [=============>................] - ETA: 1s - loss: 0.0498 - accuracy: 0.9855 851/1688 [==============>...............] - ETA: 1s - loss: 0.0497 - accuracy: 0.9855 876/1688 [==============>...............] - ETA: 1s - loss: 0.0496 - accuracy: 0.9855 900/1688 [==============>...............] - ETA: 1s - loss: 0.0496 - accuracy: 0.9855 924/1688 [===============>..............] - ETA: 1s - loss: 0.0495 - accuracy: 0.9855 948/1688 [===============>..............] - ETA: 1s - loss: 0.0494 - accuracy: 0.9855 972/1688 [================>.............] - ETA: 1s - loss: 0.0494 - accuracy: 0.9856 996/1688 [================>.............] - ETA: 1s - loss: 0.0493 - accuracy: 0.9856 1021/1688 [=================>............] - ETA: 1s - loss: 0.0493 - accuracy: 0.9856 1045/1688 [=================>............] - ETA: 1s - loss: 0.0493 - accuracy: 0.9856 1069/1688 [=================>............] - ETA: 1s - loss: 0.0492 - accuracy: 0.9856 1093/1688 [==================>...........] - ETA: 1s - loss: 0.0492 - accuracy: 0.9856 1117/1688 [==================>...........] - ETA: 1s - loss: 0.0492 - accuracy: 0.9856 1142/1688 [===================>..........] - ETA: 1s - loss: 0.0491 - accuracy: 0.9856 1167/1688 [===================>..........] - ETA: 1s - loss: 0.0491 - accuracy: 0.9856 1192/1688 [====================>.........] - ETA: 1s - loss: 0.0491 - accuracy: 0.9856 1217/1688 [====================>.........] - ETA: 0s - loss: 0.0491 - accuracy: 0.9856 1242/1688 [=====================>........] - ETA: 0s - loss: 0.0491 - accuracy: 0.9856 1267/1688 [=====================>........] - ETA: 0s - loss: 0.0490 - accuracy: 0.9856 1292/1688 [=====================>........] - ETA: 0s - loss: 0.0490 - accuracy: 0.9856 1316/1688 [======================>.......] - ETA: 0s - loss: 0.0490 - accuracy: 0.9856 1339/1688 [======================>.......] - ETA: 0s - loss: 0.0490 - accuracy: 0.9856 1364/1688 [=======================>......] - ETA: 0s - loss: 0.0490 - accuracy: 0.9856 1389/1688 [=======================>......] - ETA: 0s - loss: 0.0490 - accuracy: 0.9856 1413/1688 [========================>.....] - ETA: 0s - loss: 0.0489 - accuracy: 0.9856 1437/1688 [========================>.....] - ETA: 0s - loss: 0.0489 - accuracy: 0.9856 1460/1688 [========================>.....] - ETA: 0s - loss: 0.0489 - accuracy: 0.9856 1484/1688 [=========================>....] - ETA: 0s - loss: 0.0489 - accuracy: 0.9856 1508/1688 [=========================>....] - ETA: 0s - loss: 0.0489 - accuracy: 0.9856 1532/1688 [==========================>...] - ETA: 0s - loss: 0.0489 - accuracy: 0.9856 1557/1688 [==========================>...] - ETA: 0s - loss: 0.0489 - accuracy: 0.9856 1581/1688 [===========================>..] - ETA: 0s - loss: 0.0489 - accuracy: 0.9856 1606/1688 [===========================>..] - ETA: 0s - loss: 0.0489 - accuracy: 0.9856 1630/1688 [===========================>..] - ETA: 0s - loss: 0.0489 - accuracy: 0.9856 1654/1688 [============================>.] - ETA: 0s - loss: 0.0489 - accuracy: 0.9856 1678/1688 [============================>.] - ETA: 0s - loss: 0.0488 - accuracy: 0.9856 1688/1688 [==============================] - 4s 2ms/step - loss: 0.0488 - accuracy: 0.9856 - val_loss: 0.0562 - val_accuracy: 0.9855 Epoch 8/10 1/1688 [..............................] - ETA: 4s - loss: 0.0077 - accuracy: 1.0000 26/1688 [..............................] - ETA: 3s - loss: 0.0257 - accuracy: 0.9913 51/1688 [..............................] - ETA: 3s - loss: 0.0308 - accuracy: 0.9903 75/1688 [>.............................] - ETA: 3s - loss: 0.0342 - accuracy: 0.9895 99/1688 [>.............................] - ETA: 3s - loss: 0.0368 - accuracy: 0.9887 123/1688 [=>............................] - ETA: 3s - loss: 0.0387 - accuracy: 0.9883 147/1688 [=>............................] - ETA: 3s - loss: 0.0400 - accuracy: 0.9879 172/1688 [==>...........................] - ETA: 3s - loss: 0.0410 - accuracy: 0.9875 197/1688 [==>...........................] - ETA: 3s - loss: 0.0417 - accuracy: 0.9873 221/1688 [==>...........................] - ETA: 3s - loss: 0.0422 - accuracy: 0.9871 246/1688 [===>..........................] - ETA: 2s - loss: 0.0426 - accuracy: 0.9869 270/1688 [===>..........................] - ETA: 2s - loss: 0.0429 - accuracy: 0.9867 294/1688 [====>.........................] - ETA: 2s - loss: 0.0431 - accuracy: 0.9866 318/1688 [====>.........................] - ETA: 2s - loss: 0.0432 - accuracy: 0.9866 342/1688 [=====>........................] - ETA: 2s - loss: 0.0433 - accuracy: 0.9865 367/1688 [=====>........................] - ETA: 2s - loss: 0.0433 - accuracy: 0.9865 391/1688 [=====>........................] - ETA: 2s - loss: 0.0433 - accuracy: 0.9864 415/1688 [======>.......................] - ETA: 2s - loss: 0.0433 - accuracy: 0.9864 440/1688 [======>.......................] - ETA: 2s - loss: 0.0433 - accuracy: 0.9864 465/1688 [=======>......................] - ETA: 2s - loss: 0.0433 - accuracy: 0.9863 489/1688 [=======>......................] - ETA: 2s - loss: 0.0434 - accuracy: 0.9863 513/1688 [========>.....................] - ETA: 2s - loss: 0.0434 - accuracy: 0.9863 537/1688 [========>.....................] - ETA: 2s - loss: 0.0435 - accuracy: 0.9863 562/1688 [========>.....................] - ETA: 2s - loss: 0.0435 - accuracy: 0.9862 587/1688 [=========>....................] - ETA: 2s - loss: 0.0435 - accuracy: 0.9863 612/1688 [=========>....................] - ETA: 2s - loss: 0.0435 - accuracy: 0.9863 636/1688 [==========>...................] - ETA: 2s - loss: 0.0434 - accuracy: 0.9863 659/1688 [==========>...................] - ETA: 2s - loss: 0.0434 - accuracy: 0.9863 684/1688 [===========>..................] - ETA: 2s - loss: 0.0433 - accuracy: 0.9863 708/1688 [===========>..................] - ETA: 2s - loss: 0.0432 - accuracy: 0.9864 732/1688 [============>.................] - ETA: 2s - loss: 0.0432 - accuracy: 0.9864 756/1688 [============>.................] - ETA: 1s - loss: 0.0431 - accuracy: 0.9864 780/1688 [============>.................] - ETA: 1s - loss: 0.0431 - accuracy: 0.9864 803/1688 [=============>................] - ETA: 1s - loss: 0.0431 - accuracy: 0.9864 826/1688 [=============>................] - ETA: 1s - loss: 0.0430 - accuracy: 0.9865 850/1688 [==============>...............] - ETA: 1s - loss: 0.0430 - accuracy: 0.9865 874/1688 [==============>...............] - ETA: 1s - loss: 0.0430 - accuracy: 0.9865 899/1688 [==============>...............] - ETA: 1s - loss: 0.0430 - accuracy: 0.9865 924/1688 [===============>..............] - ETA: 1s - loss: 0.0429 - accuracy: 0.9865 948/1688 [===============>..............] - ETA: 1s - loss: 0.0429 - accuracy: 0.9865 973/1688 [================>.............] - ETA: 1s - loss: 0.0429 - accuracy: 0.9865 996/1688 [================>.............] - ETA: 1s - loss: 0.0429 - accuracy: 0.9866 1020/1688 [=================>............] - ETA: 1s - loss: 0.0429 - accuracy: 0.9866 1044/1688 [=================>............] - ETA: 1s - loss: 0.0429 - accuracy: 0.9866 1068/1688 [=================>............] - ETA: 1s - loss: 0.0428 - accuracy: 0.9866 1093/1688 [==================>...........] - ETA: 1s - loss: 0.0428 - accuracy: 0.9866 1117/1688 [==================>...........] - ETA: 1s - loss: 0.0428 - accuracy: 0.9866 1141/1688 [===================>..........] - ETA: 1s - loss: 0.0428 - accuracy: 0.9866 1165/1688 [===================>..........] - ETA: 1s - loss: 0.0428 - accuracy: 0.9866 1188/1688 [====================>.........] - ETA: 1s - loss: 0.0427 - accuracy: 0.9866 1212/1688 [====================>.........] - ETA: 1s - loss: 0.0427 - accuracy: 0.9866 1237/1688 [====================>.........] - ETA: 0s - loss: 0.0427 - accuracy: 0.9867 1261/1688 [=====================>........] - ETA: 0s - loss: 0.0427 - accuracy: 0.9867 1285/1688 [=====================>........] - ETA: 0s - loss: 0.0427 - accuracy: 0.9867 1309/1688 [======================>.......] - ETA: 0s - loss: 0.0427 - accuracy: 0.9867 1332/1688 [======================>.......] - ETA: 0s - loss: 0.0427 - accuracy: 0.9867 1356/1688 [=======================>......] - ETA: 0s - loss: 0.0427 - accuracy: 0.9867 1381/1688 [=======================>......] - ETA: 0s - loss: 0.0427 - accuracy: 0.9867 1405/1688 [=======================>......] - ETA: 0s - loss: 0.0427 - accuracy: 0.9867 1430/1688 [========================>.....] - ETA: 0s - loss: 0.0427 - accuracy: 0.9867 1454/1688 [========================>.....] - ETA: 0s - loss: 0.0427 - accuracy: 0.9867 1478/1688 [=========================>....] - ETA: 0s - loss: 0.0427 - accuracy: 0.9867 1501/1688 [=========================>....] - ETA: 0s - loss: 0.0427 - accuracy: 0.9867 1525/1688 [==========================>...] - ETA: 0s - loss: 0.0427 - accuracy: 0.9867 1550/1688 [==========================>...] - ETA: 0s - loss: 0.0428 - accuracy: 0.9867 1574/1688 [==========================>...] - ETA: 0s - loss: 0.0428 - accuracy: 0.9867 1598/1688 [===========================>..] - ETA: 0s - loss: 0.0428 - accuracy: 0.9867 1622/1688 [===========================>..] - ETA: 0s - loss: 0.0428 - accuracy: 0.9867 1648/1688 [============================>.] - ETA: 0s - loss: 0.0428 - accuracy: 0.9867 1672/1688 [============================>.] - ETA: 0s - loss: 0.0428 - accuracy: 0.9867 1688/1688 [==============================] - 4s 2ms/step - loss: 0.0428 - accuracy: 0.9867 - val_loss: 0.0646 - val_accuracy: 0.9808 Epoch 9/10 1/1688 [..............................] - ETA: 4s - loss: 0.0459 - accuracy: 0.9688 25/1688 [..............................] - ETA: 3s - loss: 0.0254 - accuracy: 0.9903 49/1688 [..............................] - ETA: 3s - loss: 0.0256 - accuracy: 0.9908 74/1688 [>.............................] - ETA: 3s - loss: 0.0261 - accuracy: 0.9910 99/1688 [>.............................] - ETA: 3s - loss: 0.0272 - accuracy: 0.9909 124/1688 [=>............................] - ETA: 3s - loss: 0.0281 - accuracy: 0.9908 148/1688 [=>............................] - ETA: 3s - loss: 0.0286 - accuracy: 0.9908 171/1688 [==>...........................] - ETA: 3s - loss: 0.0290 - accuracy: 0.9907 196/1688 [==>...........................] - ETA: 3s - loss: 0.0292 - accuracy: 0.9907 221/1688 [==>...........................] - ETA: 3s - loss: 0.0293 - accuracy: 0.9907 246/1688 [===>..........................] - ETA: 3s - loss: 0.0295 - accuracy: 0.9906 270/1688 [===>..........................] - ETA: 2s - loss: 0.0298 - accuracy: 0.9906 295/1688 [====>.........................] - ETA: 2s - loss: 0.0301 - accuracy: 0.9906 320/1688 [====>.........................] - ETA: 2s - loss: 0.0305 - accuracy: 0.9905 344/1688 [=====>........................] - ETA: 2s - loss: 0.0309 - accuracy: 0.9905 368/1688 [=====>........................] - ETA: 2s - loss: 0.0312 - accuracy: 0.9904 393/1688 [=====>........................] - ETA: 2s - loss: 0.0316 - accuracy: 0.9904 417/1688 [======>.......................] - ETA: 2s - loss: 0.0319 - accuracy: 0.9903 441/1688 [======>.......................] - ETA: 2s - loss: 0.0323 - accuracy: 0.9903 466/1688 [=======>......................] - ETA: 2s - loss: 0.0326 - accuracy: 0.9902 491/1688 [=======>......................] - ETA: 2s - loss: 0.0329 - accuracy: 0.9901 516/1688 [========>.....................] - ETA: 2s - loss: 0.0332 - accuracy: 0.9900 541/1688 [========>.....................] - ETA: 2s - loss: 0.0335 - accuracy: 0.9900 566/1688 [=========>....................] - ETA: 2s - loss: 0.0337 - accuracy: 0.9899 591/1688 [=========>....................] - ETA: 2s - loss: 0.0340 - accuracy: 0.9899 617/1688 [=========>....................] - ETA: 2s - loss: 0.0342 - accuracy: 0.9898 643/1688 [==========>...................] - ETA: 2s - loss: 0.0344 - accuracy: 0.9897 668/1688 [==========>...................] - ETA: 2s - loss: 0.0346 - accuracy: 0.9897 692/1688 [===========>..................] - ETA: 2s - loss: 0.0348 - accuracy: 0.9896 716/1688 [===========>..................] - ETA: 2s - loss: 0.0349 - accuracy: 0.9896 740/1688 [============>.................] - ETA: 1s - loss: 0.0351 - accuracy: 0.9896 764/1688 [============>.................] - ETA: 1s - loss: 0.0352 - accuracy: 0.9895 790/1688 [=============>................] - ETA: 1s - loss: 0.0353 - accuracy: 0.9895 814/1688 [=============>................] - ETA: 1s - loss: 0.0355 - accuracy: 0.9895 839/1688 [=============>................] - ETA: 1s - loss: 0.0356 - accuracy: 0.9894 863/1688 [==============>...............] - ETA: 1s - loss: 0.0357 - accuracy: 0.9894 887/1688 [==============>...............] - ETA: 1s - loss: 0.0358 - accuracy: 0.9894 911/1688 [===============>..............] - ETA: 1s - loss: 0.0359 - accuracy: 0.9893 935/1688 [===============>..............] - ETA: 1s - loss: 0.0360 - accuracy: 0.9893 959/1688 [================>.............] - ETA: 1s - loss: 0.0361 - accuracy: 0.9893 984/1688 [================>.............] - ETA: 1s - loss: 0.0362 - accuracy: 0.9892 1009/1688 [================>.............] - ETA: 1s - loss: 0.0363 - accuracy: 0.9892 1033/1688 [=================>............] - ETA: 1s - loss: 0.0364 - accuracy: 0.9892 1058/1688 [=================>............] - ETA: 1s - loss: 0.0365 - accuracy: 0.9892 1083/1688 [==================>...........] - ETA: 1s - loss: 0.0366 - accuracy: 0.9891 1107/1688 [==================>...........] - ETA: 1s - loss: 0.0367 - accuracy: 0.9891 1131/1688 [===================>..........] - ETA: 1s - loss: 0.0367 - accuracy: 0.9891 1155/1688 [===================>..........] - ETA: 1s - loss: 0.0368 - accuracy: 0.9891 1179/1688 [===================>..........] - ETA: 1s - loss: 0.0369 - accuracy: 0.9890 1203/1688 [====================>.........] - ETA: 1s - loss: 0.0369 - accuracy: 0.9890 1226/1688 [====================>.........] - ETA: 0s - loss: 0.0370 - accuracy: 0.9890 1250/1688 [=====================>........] - ETA: 0s - loss: 0.0370 - accuracy: 0.9890 1275/1688 [=====================>........] - ETA: 0s - loss: 0.0371 - accuracy: 0.9890 1300/1688 [======================>.......] - ETA: 0s - loss: 0.0372 - accuracy: 0.9889 1324/1688 [======================>.......] - ETA: 0s - loss: 0.0372 - accuracy: 0.9889 1348/1688 [======================>.......] - ETA: 0s - loss: 0.0373 - accuracy: 0.9889 1372/1688 [=======================>......] - ETA: 0s - loss: 0.0373 - accuracy: 0.9889 1396/1688 [=======================>......] - ETA: 0s - loss: 0.0374 - accuracy: 0.9889 1420/1688 [========================>.....] - ETA: 0s - loss: 0.0375 - accuracy: 0.9889 1445/1688 [========================>.....] - ETA: 0s - loss: 0.0375 - accuracy: 0.9888 1470/1688 [=========================>....] - ETA: 0s - loss: 0.0376 - accuracy: 0.9888 1495/1688 [=========================>....] - ETA: 0s - loss: 0.0376 - accuracy: 0.9888 1520/1688 [==========================>...] - ETA: 0s - loss: 0.0377 - accuracy: 0.9888 1545/1688 [==========================>...] - ETA: 0s - loss: 0.0377 - accuracy: 0.9888 1569/1688 [==========================>...] - ETA: 0s - loss: 0.0378 - accuracy: 0.9887 1593/1688 [===========================>..] - ETA: 0s - loss: 0.0378 - accuracy: 0.9887 1617/1688 [===========================>..] - ETA: 0s - loss: 0.0378 - accuracy: 0.9887 1641/1688 [============================>.] - ETA: 0s - loss: 0.0379 - accuracy: 0.9887 1665/1688 [============================>.] - ETA: 0s - loss: 0.0379 - accuracy: 0.9887 1688/1688 [==============================] - 4s 2ms/step - loss: 0.0379 - accuracy: 0.9887 - val_loss: 0.0617 - val_accuracy: 0.9848 Epoch 10/10 1/1688 [..............................] - ETA: 4s - loss: 0.0455 - accuracy: 0.9688 25/1688 [..............................] - ETA: 3s - loss: 0.0315 - accuracy: 0.9888 48/1688 [..............................] - ETA: 3s - loss: 0.0318 - accuracy: 0.9902 71/1688 [>.............................] - ETA: 3s - loss: 0.0314 - accuracy: 0.9907 95/1688 [>.............................] - ETA: 3s - loss: 0.0311 - accuracy: 0.9910 119/1688 [=>............................] - ETA: 3s - loss: 0.0309 - accuracy: 0.9912 143/1688 [=>............................] - ETA: 3s - loss: 0.0307 - accuracy: 0.9913 167/1688 [=>............................] - ETA: 3s - loss: 0.0305 - accuracy: 0.9914 191/1688 [==>...........................] - ETA: 3s - loss: 0.0302 - accuracy: 0.9915 215/1688 [==>...........................] - ETA: 3s - loss: 0.0301 - accuracy: 0.9916 239/1688 [===>..........................] - ETA: 3s - loss: 0.0301 - accuracy: 0.9917 263/1688 [===>..........................] - ETA: 3s - loss: 0.0302 - accuracy: 0.9917 287/1688 [====>.........................] - ETA: 2s - loss: 0.0302 - accuracy: 0.9917 311/1688 [====>.........................] - ETA: 2s - loss: 0.0303 - accuracy: 0.9917 336/1688 [====>.........................] - ETA: 2s - loss: 0.0303 - accuracy: 0.9917 360/1688 [=====>........................] - ETA: 2s - loss: 0.0303 - accuracy: 0.9917 383/1688 [=====>........................] - ETA: 2s - loss: 0.0304 - accuracy: 0.9917 407/1688 [======>.......................] - ETA: 2s - loss: 0.0304 - accuracy: 0.9917 432/1688 [======>.......................] - ETA: 2s - loss: 0.0305 - accuracy: 0.9917 456/1688 [=======>......................] - ETA: 2s - loss: 0.0305 - accuracy: 0.9917 481/1688 [=======>......................] - ETA: 2s - loss: 0.0306 - accuracy: 0.9916 505/1688 [=======>......................] - ETA: 2s - loss: 0.0306 - accuracy: 0.9916 529/1688 [========>.....................] - ETA: 2s - loss: 0.0307 - accuracy: 0.9916 553/1688 [========>.....................] - ETA: 2s - loss: 0.0307 - accuracy: 0.9916 578/1688 [=========>....................] - ETA: 2s - loss: 0.0308 - accuracy: 0.9916 603/1688 [=========>....................] - ETA: 2s - loss: 0.0309 - accuracy: 0.9916 627/1688 [==========>...................] - ETA: 2s - loss: 0.0310 - accuracy: 0.9916 652/1688 [==========>...................] - ETA: 2s - loss: 0.0311 - accuracy: 0.9915 677/1688 [===========>..................] - ETA: 2s - loss: 0.0311 - accuracy: 0.9915 702/1688 [===========>..................] - ETA: 2s - loss: 0.0312 - accuracy: 0.9915 726/1688 [===========>..................] - ETA: 2s - loss: 0.0313 - accuracy: 0.9915 750/1688 [============>.................] - ETA: 1s - loss: 0.0314 - accuracy: 0.9914 775/1688 [============>.................] - ETA: 1s - loss: 0.0315 - accuracy: 0.9914 800/1688 [=============>................] - ETA: 1s - loss: 0.0316 - accuracy: 0.9914 824/1688 [=============>................] - ETA: 1s - loss: 0.0317 - accuracy: 0.9913 848/1688 [==============>...............] - ETA: 1s - loss: 0.0318 - accuracy: 0.9913 872/1688 [==============>...............] - ETA: 1s - loss: 0.0319 - accuracy: 0.9913 896/1688 [==============>...............] - ETA: 1s - loss: 0.0319 - accuracy: 0.9913 921/1688 [===============>..............] - ETA: 1s - loss: 0.0320 - accuracy: 0.9912 946/1688 [===============>..............] - ETA: 1s - loss: 0.0321 - accuracy: 0.9912 971/1688 [================>.............] - ETA: 1s - loss: 0.0322 - accuracy: 0.9912 996/1688 [================>.............] - ETA: 1s - loss: 0.0322 - accuracy: 0.9911 1021/1688 [=================>............] - ETA: 1s - loss: 0.0323 - accuracy: 0.9911 1046/1688 [=================>............] - ETA: 1s - loss: 0.0323 - accuracy: 0.9911 1069/1688 [=================>............] - ETA: 1s - loss: 0.0324 - accuracy: 0.9911 1093/1688 [==================>...........] - ETA: 1s - loss: 0.0325 - accuracy: 0.9910 1117/1688 [==================>...........] - ETA: 1s - loss: 0.0325 - accuracy: 0.9910 1142/1688 [===================>..........] - ETA: 1s - loss: 0.0326 - accuracy: 0.9910 1166/1688 [===================>..........] - ETA: 1s - loss: 0.0327 - accuracy: 0.9909 1191/1688 [====================>.........] - ETA: 1s - loss: 0.0327 - accuracy: 0.9909 1216/1688 [====================>.........] - ETA: 0s - loss: 0.0328 - accuracy: 0.9909 1241/1688 [=====================>........] - ETA: 0s - loss: 0.0328 - accuracy: 0.9909 1266/1688 [=====================>........] - ETA: 0s - loss: 0.0329 - accuracy: 0.9908 1291/1688 [=====================>........] - ETA: 0s - loss: 0.0330 - accuracy: 0.9908 1315/1688 [======================>.......] - ETA: 0s - loss: 0.0330 - accuracy: 0.9908 1339/1688 [======================>.......] - ETA: 0s - loss: 0.0331 - accuracy: 0.9908 1363/1688 [=======================>......] - ETA: 0s - loss: 0.0331 - accuracy: 0.9907 1387/1688 [=======================>......] - ETA: 0s - loss: 0.0332 - accuracy: 0.9907 1411/1688 [========================>.....] - ETA: 0s - loss: 0.0333 - accuracy: 0.9907 1435/1688 [========================>.....] - ETA: 0s - loss: 0.0333 - accuracy: 0.9907 1460/1688 [========================>.....] - ETA: 0s - loss: 0.0334 - accuracy: 0.9907 1484/1688 [=========================>....] - ETA: 0s - loss: 0.0334 - accuracy: 0.9906 1508/1688 [=========================>....] - ETA: 0s - loss: 0.0335 - accuracy: 0.9906 1533/1688 [==========================>...] - ETA: 0s - loss: 0.0335 - accuracy: 0.9906 1558/1688 [==========================>...] - ETA: 0s - loss: 0.0336 - accuracy: 0.9906 1581/1688 [===========================>..] - ETA: 0s - loss: 0.0336 - accuracy: 0.9905 1606/1688 [===========================>..] - ETA: 0s - loss: 0.0337 - accuracy: 0.9905 1630/1688 [===========================>..] - ETA: 0s - loss: 0.0337 - accuracy: 0.9905 1655/1688 [============================>.] - ETA: 0s - loss: 0.0338 - accuracy: 0.9905 1679/1688 [============================>.] - ETA: 0s - loss: 0.0338 - accuracy: 0.9904 1688/1688 [==============================] - 4s 2ms/step - loss: 0.0338 - accuracy: 0.9904 - val_loss: 0.0553 - val_accuracy: 0.9853 <tensorflow.python.keras.callbacks.History at 0x7fde60ea4748>
기준 모델을 평가하고 나중에 사용할 수 있도록 저장하기
_, baseline_model_accuracy = model.evaluate(
test_images, test_labels, verbose=0)
print('Baseline test accuracy:', baseline_model_accuracy)
_, keras_file = tempfile.mkstemp('.h5')
print('Saving model to: ', keras_file)
tf.keras.models.save_model(model, keras_file, include_optimizer=False)
Baseline test accuracy: 0.9814000129699707 Saving model to: /tmp/tmpaxhvi0yg.h5
클러스터링을 사용하여 사전 훈련된 모델 미세 조정하기
사전 훈련된 전체 모델에 cluster_weights() API를 적용하여 압축 후 적절한 정확성을 유지하면서 모델 크기가 줄어드는 효과를 입증합니다. 해당 사용 사례에서 정확성과 압축률의 균형을 가장 잘 유지하는 방법은 포괄적 가이드의 레이어별 예를 참조하세요.
모델 정의 및 클러스터링 API 적용하기
모델을 클러스터링 API로 전달하기 전에 모델이 훈련되었고 수용 가능한 정확성을 보이는지 확인합니다.
import tensorflow_model_optimization as tfmot
cluster_weights = tfmot.clustering.keras.cluster_weights
CentroidInitialization = tfmot.clustering.keras.CentroidInitialization
clustering_params = {
'number_of_clusters': 16,
'cluster_centroids_init': CentroidInitialization.LINEAR
}
# Cluster a whole model
clustered_model = cluster_weights(model, **clustering_params)
# Use smaller learning rate for fine-tuning clustered model
opt = tf.keras.optimizers.Adam(learning_rate=1e-5)
clustered_model.compile(
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=opt,
metrics=['accuracy'])
clustered_model.summary()
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= cluster_reshape (ClusterWeig (None, 28, 28, 1) 0 _________________________________________________________________ cluster_conv2d (ClusterWeigh (None, 26, 26, 12) 136 _________________________________________________________________ cluster_max_pooling2d (Clust (None, 13, 13, 12) 0 _________________________________________________________________ cluster_flatten (ClusterWeig (None, 2028) 0 _________________________________________________________________ cluster_dense (ClusterWeight (None, 10) 20306 ================================================================= Total params: 20,442 Trainable params: 54 Non-trainable params: 20,388 _________________________________________________________________
모델을 미세 조정하고 기준 대비 정확성 평가하기
하나의 epoch 동안 클러스터링이 있는 모델을 미세 조정합니다.
# Fine-tune model
clustered_model.fit(
train_images,
train_labels,
batch_size=500,
epochs=1,
validation_split=0.1)
1/108 [..............................] - ETA: 1:01 - loss: 0.0298 - accuracy: 0.9920 16/108 [===>..........................] - ETA: 0s - loss: 0.0411 - accuracy: 0.9851 31/108 [=======>......................] - ETA: 0s - loss: 0.0414 - accuracy: 0.9849 47/108 [============>.................] - ETA: 0s - loss: 0.0412 - accuracy: 0.9851 62/108 [================>.............] - ETA: 0s - loss: 0.0415 - accuracy: 0.9851 78/108 [====================>.........] - ETA: 0s - loss: 0.0418 - accuracy: 0.9850 94/108 [=========================>....] - ETA: 0s - loss: 0.0421 - accuracy: 0.9851 108/108 [==============================] - 1s 5ms/step - loss: 0.0423 - accuracy: 0.9851 - val_loss: 0.0644 - val_accuracy: 0.9828 <tensorflow.python.keras.callbacks.History at 0x7fdecf7e9b38>
이 예의 경우, 기준과 비교하여 클러스터링 후 테스트 정확성의 손실이 미미합니다.
_, clustered_model_accuracy = clustered_model.evaluate(
test_images, test_labels, verbose=0)
print('Baseline test accuracy:', baseline_model_accuracy)
print('Clustered test accuracy:', clustered_model_accuracy)
Baseline test accuracy: 0.9814000129699707 Clustered test accuracy: 0.9786999821662903
클러스터링으로부터 6배 더 작은 모델 만들기
클러스터링의 압축 이점을 확인하려면 strip_clustering과 표준 압축 알고리즘(예: gzip 이용) 적용이 모두 필요합니다.
먼저, TensorFlow를 위한 압축 가능한 모델을 만듭니다. 여기서, strip_clustering은 훈련 중에만 클러스터링에 필요한 모든 변수(예: 클러스터 중심과 인덱스를 저장하기 위한 tf.Variable)를 제거합니다. 이러한 변수를 제거하지 않으면 추론 중에 모델 크기가 증가하게 됩니다.
final_model = tfmot.clustering.keras.strip_clustering(clustered_model)
_, clustered_keras_file = tempfile.mkstemp('.h5')
print('Saving clustered model to: ', clustered_keras_file)
tf.keras.models.save_model(final_model, clustered_keras_file,
include_optimizer=False)
Saving clustered model to: /tmp/tmpvjj70etk.h5
그런 다음, TFLite를 위한 압축 가능한 모델을 만듭니다. 클러스터링된 모델을 대상 백엔드에서 실행 가능한 형식으로 변환할 수 있습니다. TensorFlow Lite는 모바일 기기에 배포하는 데 사용할 수 있는 예입니다.
clustered_tflite_file = '/tmp/clustered_mnist.tflite'
converter = tf.lite.TFLiteConverter.from_keras_model(final_model)
tflite_clustered_model = converter.convert()
with open(clustered_tflite_file, 'wb') as f:
f.write(tflite_clustered_model)
print('Saved clustered TFLite model to:', clustered_tflite_file)
INFO:tensorflow:Assets written to: /tmp/tmpg3n3nbv4/assets Saved clustered TFLite model to: /tmp/clustered_mnist.tflite
실제로 gzip을 통해 모델을 압축하는 도우미 함수를 정의하고 압축된 크기를 측정합니다.
def get_gzipped_model_size(file):
# It returns the size of the gzipped model in bytes.
import os
import zipfile
_, zipped_file = tempfile.mkstemp('.zip')
with zipfile.ZipFile(zipped_file, 'w', compression=zipfile.ZIP_DEFLATED) as f:
f.write(file)
return os.path.getsize(zipped_file)
클러스터링으로부터 모델이 6배 더 작아진 것을 확인하세요.
print("Size of gzipped baseline Keras model: %.2f bytes" % (get_gzipped_model_size(keras_file)))
print("Size of gzipped clustered Keras model: %.2f bytes" % (get_gzipped_model_size(clustered_keras_file)))
print("Size of gzipped clustered TFlite model: %.2f bytes" % (get_gzipped_model_size(clustered_tflite_file)))
Size of gzipped baseline Keras model: 77991.00 bytes Size of gzipped clustered Keras model: 12656.00 bytes Size of gzipped clustered TFlite model: 12178.00 bytes
가중치 클러스터링과 훈련 후 양자화를 결합하여 8배 더 작은 TFLite 모델 만들기
추가적인 이점을 얻기 위해 클러스터링한 모델에 훈련 후 양자화를 적용할 수 있습니다.
converter = tf.lite.TFLiteConverter.from_keras_model(final_model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_quant_model = converter.convert()
_, quantized_and_clustered_tflite_file = tempfile.mkstemp('.tflite')
with open(quantized_and_clustered_tflite_file, 'wb') as f:
f.write(tflite_quant_model)
print('Saved quantized and clustered TFLite model to:', quantized_and_clustered_tflite_file)
print("Size of gzipped baseline Keras model: %.2f bytes" % (get_gzipped_model_size(keras_file)))
print("Size of gzipped clustered and quantized TFlite model: %.2f bytes" % (get_gzipped_model_size(quantized_and_clustered_tflite_file)))
INFO:tensorflow:Assets written to: /tmp/tmprx_pw7t5/assets INFO:tensorflow:Assets written to: /tmp/tmprx_pw7t5/assets Saved quantized and clustered TFLite model to: /tmp/tmpq1ux2lbh.tflite Size of gzipped baseline Keras model: 77991.00 bytes Size of gzipped clustered and quantized TFlite model: 9335.00 bytes
TF에서 TFLite로 정확성이 지속되는지 확인하기
테스트 데이터세트에서 TFLite 모델을 평가하는 도우미 함수를 정의합니다.
def eval_model(interpreter):
input_index = interpreter.get_input_details()[0]["index"]
output_index = interpreter.get_output_details()[0]["index"]
# Run predictions on every image in the "test" dataset.
prediction_digits = []
for i, test_image in enumerate(test_images):
if i % 1000 == 0:
print('Evaluated on {n} results so far.'.format(n=i))
# Pre-processing: add batch dimension and convert to float32 to match with
# the model's input data format.
test_image = np.expand_dims(test_image, axis=0).astype(np.float32)
interpreter.set_tensor(input_index, test_image)
# Run inference.
interpreter.invoke()
# Post-processing: remove batch dimension and find the digit with highest
# probability.
output = interpreter.tensor(output_index)
digit = np.argmax(output()[0])
prediction_digits.append(digit)
print('\n')
# Compare prediction results with ground truth labels to calculate accuracy.
prediction_digits = np.array(prediction_digits)
accuracy = (prediction_digits == test_labels).mean()
return accuracy
클러스터링되고 양자화된 모델을 평가한 다음, TensorFlow의 정확성이 TFLite 백엔드까지 유지되는지 확인합니다.
interpreter = tf.lite.Interpreter(model_content=tflite_quant_model)
interpreter.allocate_tensors()
test_accuracy = eval_model(interpreter)
print('Clustered and quantized TFLite test_accuracy:', test_accuracy)
print('Clustered TF test accuracy:', clustered_model_accuracy)
Evaluated on 0 results so far. Evaluated on 1000 results so far. Evaluated on 2000 results so far. Evaluated on 3000 results so far. Evaluated on 4000 results so far. Evaluated on 5000 results so far. Evaluated on 6000 results so far. Evaluated on 7000 results so far. Evaluated on 8000 results so far. Evaluated on 9000 results so far. Clustered and quantized TFLite test_accuracy: 0.9789 Clustered TF test accuracy: 0.9786999821662903
결론
이 튜토리얼에서는 TensorFlow 모델 최적화 도구 키트 API를 사용하여 클러스터링된 모델을 만드는 방법을 알아보았습니다. 구체적으로, 정확성 차이를 최소화하면서 8배 더 작은 MNIST용 모델을 생성하기 위한 엔드 투 엔드 예제를 살펴보았습니다. 리소스가 제한된 환경에서 배포할 때 특히 중요할 수 있는 이 새로운 기능을 한 번 사용해 보세요.