 在 TensorFlow.org 上查看 |
 在 Google Colab 中运行 |
 在 GitHub 中查看源代码 |
{img1下载笔记本 |
概述
此笔记本将演示如何使用 Addons 包中的 Lazy Adam 优化器。
LazyAdam
LazyAdam 是 Adam 优化器的一种变体，可以更高效地处理稀疏更新。原始的 Adam 算法为每个可训练变量维护两个移动平均累加器，这些累加器在每一步都会更新。此类为稀疏变量提供了更加懒惰的梯度更新处理。它仅更新当前批次中出现的稀疏变量索引的移动平均累加器，而不是更新所有索引的累加器。与原始的 Adam 优化器相比，它可以大幅提高某些应用的模型训练吞吐量。但是，它的语义与原始的 Adam 算法略有不同，这可能会产生不同的实验结果。
设置
pip install -q -U tensorflow-addonsimport tensorflow as tf
import tensorflow_addons as tfa
# Hyperparameters
batch_size=64
epochs=10
构建模型
model = tf.keras.Sequential([
tf.keras.layers.Dense(64, input_shape=(784,), activation='relu', name='dense_1'),
tf.keras.layers.Dense(64, activation='relu', name='dense_2'),
tf.keras.layers.Dense(10, activation='softmax', name='predictions'),
])
准备数据
# Load MNIST dataset as NumPy arrays
dataset = {}
num_validation = 10000
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
# Preprocess the data
x_train = x_train.reshape(-1, 784).astype('float32') / 255
x_test = x_test.reshape(-1, 784).astype('float32') / 255
训练和评估
只需用新的 TFA 优化器替换典型的 Keras 优化器
# Compile the model
model.compile(
optimizer=tfa.optimizers.LazyAdam(0.001), # Utilize TFA optimizer
loss=tf.keras.losses.SparseCategoricalCrossentropy(),
metrics=['accuracy'])
# Train the network
history = model.fit(
x_train,
y_train,
batch_size=batch_size,
epochs=epochs)
Epoch 1/10 1/938 [..............................] - ETA: 0s - loss: 2.2949 - accuracy: 0.1250 31/938 [..............................] - ETA: 1s - loss: 1.7769 - accuracy: 0.4819 61/938 [>.............................] - ETA: 1s - loss: 1.3210 - accuracy: 0.6270 90/938 [=>............................] - ETA: 1s - loss: 1.0793 - accuracy: 0.6976 121/938 [==>...........................] - ETA: 1s - loss: 0.9019 - accuracy: 0.7470 151/938 [===>..........................] - ETA: 1s - loss: 0.8015 - accuracy: 0.7763 181/938 [====>.........................] - ETA: 1s - loss: 0.7271 - accuracy: 0.7951 211/938 [=====>........................] - ETA: 1s - loss: 0.6705 - accuracy: 0.8106 242/938 [======>.......................] - ETA: 1s - loss: 0.6270 - accuracy: 0.8238 271/938 [=======>......................] - ETA: 1s - loss: 0.5887 - accuracy: 0.8343 301/938 [========>.....................] - ETA: 1s - loss: 0.5578 - accuracy: 0.8429 331/938 [=========>....................] - ETA: 1s - loss: 0.5334 - accuracy: 0.8496 361/938 [==========>...................] - ETA: 0s - loss: 0.5121 - accuracy: 0.8557 391/938 [===========>..................] - ETA: 0s - loss: 0.4916 - accuracy: 0.8611 421/938 [============>.................] - ETA: 0s - loss: 0.4734 - accuracy: 0.8662 451/938 [=============>................] - ETA: 0s - loss: 0.4590 - accuracy: 0.8700 481/938 [==============>...............] - ETA: 0s - loss: 0.4451 - accuracy: 0.8742 510/938 [===============>..............] - ETA: 0s - loss: 0.4311 - accuracy: 0.8781 540/938 [================>.............] - ETA: 0s - loss: 0.4171 - accuracy: 0.8818 569/938 [=================>............] - ETA: 0s - loss: 0.4077 - accuracy: 0.8839 600/938 [==================>...........] - ETA: 0s - loss: 0.3970 - accuracy: 0.8868 630/938 [===================>..........] - ETA: 0s - loss: 0.3877 - accuracy: 0.8892 660/938 [====================>.........] - ETA: 0s - loss: 0.3793 - accuracy: 0.8916 690/938 [=====================>........] - ETA: 0s - loss: 0.3714 - accuracy: 0.8937 720/938 [======================>.......] - ETA: 0s - loss: 0.3630 - accuracy: 0.8960 749/938 [======================>.......] - ETA: 0s - loss: 0.3559 - accuracy: 0.8979 778/938 [=======================>......] - ETA: 0s - loss: 0.3496 - accuracy: 0.8997 807/938 [========================>.....] - ETA: 0s - loss: 0.3435 - accuracy: 0.9014 837/938 [=========================>....] - ETA: 0s - loss: 0.3373 - accuracy: 0.9031 866/938 [==========================>...] - ETA: 0s - loss: 0.3308 - accuracy: 0.9048 895/938 [===========================>..] - ETA: 0s - loss: 0.3256 - accuracy: 0.9060 925/938 [============================>.] - ETA: 0s - loss: 0.3212 - accuracy: 0.9073 938/938 [==============================] - 2s 2ms/step - loss: 0.3188 - accuracy: 0.9079 Epoch 2/10 1/938 [..............................] - ETA: 0s - loss: 0.1933 - accuracy: 0.8906 31/938 [..............................] - ETA: 1s - loss: 0.1347 - accuracy: 0.9572 61/938 [>.............................] - ETA: 1s - loss: 0.1384 - accuracy: 0.9588 91/938 [=>............................] - ETA: 1s - loss: 0.1440 - accuracy: 0.9579 120/938 [==>...........................] - ETA: 1s - loss: 0.1432 - accuracy: 0.9583 149/938 [===>..........................] - ETA: 1s - loss: 0.1424 - accuracy: 0.9570 179/938 [====>.........................] - ETA: 1s - loss: 0.1451 - accuracy: 0.9567 209/938 [=====>........................] - ETA: 1s - loss: 0.1467 - accuracy: 0.9564 239/938 [======>.......................] - ETA: 1s - loss: 0.1461 - accuracy: 0.9565 269/938 [=======>......................] - ETA: 1s - loss: 0.1461 - accuracy: 0.9568 300/938 [========>.....................] - ETA: 1s - loss: 0.1455 - accuracy: 0.9569 330/938 [=========>....................] - ETA: 1s - loss: 0.1424 - accuracy: 0.9580 361/938 [==========>...................] - ETA: 0s - loss: 0.1437 - accuracy: 0.9581 391/938 [===========>..................] - ETA: 0s - loss: 0.1418 - accuracy: 0.9581 421/938 [============>.................] - ETA: 0s - loss: 0.1397 - accuracy: 0.9587 452/938 [=============>................] - ETA: 0s - loss: 0.1388 - accuracy: 0.9589 483/938 [==============>...............] - ETA: 0s - loss: 0.1385 - accuracy: 0.9590 512/938 [===============>..............] - ETA: 0s - loss: 0.1373 - accuracy: 0.9591 543/938 [================>.............] - ETA: 0s - loss: 0.1355 - accuracy: 0.9596 573/938 [=================>............] - ETA: 0s - loss: 0.1353 - accuracy: 0.9599 603/938 [==================>...........] - ETA: 0s - loss: 0.1364 - accuracy: 0.9599 633/938 [===================>..........] - ETA: 0s - loss: 0.1357 - accuracy: 0.9599 663/938 [====================>.........] - ETA: 0s - loss: 0.1355 - accuracy: 0.9600 693/938 [=====================>........] - ETA: 0s - loss: 0.1348 - accuracy: 0.9601 723/938 [======================>.......] - ETA: 0s - loss: 0.1347 - accuracy: 0.9601 754/938 [=======================>......] - ETA: 0s - loss: 0.1343 - accuracy: 0.9602 784/938 [========================>.....] - ETA: 0s - loss: 0.1332 - accuracy: 0.9604 814/938 [=========================>....] - ETA: 0s - loss: 0.1330 - accuracy: 0.9603 845/938 [==========================>...] - ETA: 0s - loss: 0.1331 - accuracy: 0.9603 876/938 [===========================>..] - ETA: 0s - loss: 0.1326 - accuracy: 0.9603 905/938 [===========================>..] - ETA: 0s - loss: 0.1320 - accuracy: 0.9605 934/938 [============================>.] - ETA: 0s - loss: 0.1316 - accuracy: 0.9607 938/938 [==============================] - 2s 2ms/step - loss: 0.1316 - accuracy: 0.9607 Epoch 3/10 1/938 [..............................] - ETA: 0s - loss: 0.0688 - accuracy: 0.9844 30/938 [..............................] - ETA: 1s - loss: 0.0817 - accuracy: 0.9755 60/938 [>.............................] - ETA: 1s - loss: 0.0831 - accuracy: 0.9747 89/938 [=>............................] - ETA: 1s - loss: 0.0912 - accuracy: 0.9733 119/938 [==>...........................] - ETA: 1s - loss: 0.0977 - accuracy: 0.9712 149/938 [===>..........................] - ETA: 1s - loss: 0.0976 - accuracy: 0.9715 179/938 [====>.........................] - ETA: 1s - loss: 0.0983 - accuracy: 0.9705 209/938 [=====>........................] - ETA: 1s - loss: 0.0994 - accuracy: 0.9702 239/938 [======>.......................] - ETA: 1s - loss: 0.1014 - accuracy: 0.9693 268/938 [=======>......................] - ETA: 1s - loss: 0.1005 - accuracy: 0.9692 298/938 [========>.....................] - ETA: 1s - loss: 0.1011 - accuracy: 0.9690 328/938 [=========>....................] - ETA: 1s - loss: 0.1008 - accuracy: 0.9689 357/938 [==========>...................] - ETA: 0s - loss: 0.1015 - accuracy: 0.9685 386/938 [===========>..................] - ETA: 0s - loss: 0.1018 - accuracy: 0.9687 415/938 [============>.................] - ETA: 0s - loss: 0.1016 - accuracy: 0.9688 444/938 [=============>................] - ETA: 0s - loss: 0.1012 - accuracy: 0.9690 474/938 [==============>...............] - ETA: 0s - loss: 0.1003 - accuracy: 0.9693 504/938 [===============>..............] - ETA: 0s - loss: 0.1000 - accuracy: 0.9692 534/938 [================>.............] - ETA: 0s - loss: 0.0993 - accuracy: 0.9694 563/938 [=================>............] - ETA: 0s - loss: 0.0990 - accuracy: 0.9695 594/938 [=================>............] - ETA: 0s - loss: 0.0993 - accuracy: 0.9694 623/938 [==================>...........] - ETA: 0s - loss: 0.0988 - accuracy: 0.9695 652/938 [===================>..........] - ETA: 0s - loss: 0.0995 - accuracy: 0.9695 681/938 [====================>.........] - ETA: 0s - loss: 0.0987 - accuracy: 0.9698 711/938 [=====================>........] - ETA: 0s - loss: 0.0990 - accuracy: 0.9698 740/938 [======================>.......] - ETA: 0s - loss: 0.0992 - accuracy: 0.9698 770/938 [=======================>......] - ETA: 0s - loss: 0.0988 - accuracy: 0.9700 800/938 [========================>.....] - ETA: 0s - loss: 0.0983 - accuracy: 0.9701 830/938 [=========================>....] - ETA: 0s - loss: 0.0976 - accuracy: 0.9704 859/938 [==========================>...] - ETA: 0s - loss: 0.0976 - accuracy: 0.9703 888/938 [===========================>..] - ETA: 0s - loss: 0.0978 - accuracy: 0.9703 918/938 [============================>.] - ETA: 0s - loss: 0.0972 - accuracy: 0.9705 938/938 [==============================] - 2s 2ms/step - loss: 0.0970 - accuracy: 0.9706 Epoch 4/10 1/938 [..............................] - ETA: 0s - loss: 0.0203 - accuracy: 1.0000 30/938 [..............................] - ETA: 1s - loss: 0.0572 - accuracy: 0.9818 61/938 [>.............................] - ETA: 1s - loss: 0.0631 - accuracy: 0.9813 90/938 [=>............................] - ETA: 1s - loss: 0.0688 - accuracy: 0.9790 120/938 [==>...........................] - ETA: 1s - loss: 0.0671 - accuracy: 0.9790 150/938 [===>..........................] - ETA: 1s - loss: 0.0688 - accuracy: 0.9791 180/938 [====>.........................] - ETA: 1s - loss: 0.0697 - accuracy: 0.9785 211/938 [=====>........................] - ETA: 1s - loss: 0.0702 - accuracy: 0.9782 241/938 [======>.......................] - ETA: 1s - loss: 0.0704 - accuracy: 0.9782 270/938 [=======>......................] - ETA: 1s - loss: 0.0712 - accuracy: 0.9780 299/938 [========>.....................] - ETA: 1s - loss: 0.0707 - accuracy: 0.9779 329/938 [=========>....................] - ETA: 1s - loss: 0.0703 - accuracy: 0.9781 359/938 [==========>...................] - ETA: 0s - loss: 0.0698 - accuracy: 0.9784 389/938 [===========>..................] - ETA: 0s - loss: 0.0719 - accuracy: 0.9779 419/938 [============>.................] - ETA: 0s - loss: 0.0719 - accuracy: 0.9777 449/938 [=============>................] - ETA: 0s - loss: 0.0727 - accuracy: 0.9773 479/938 [==============>...............] - ETA: 0s - loss: 0.0725 - accuracy: 0.9773 508/938 [===============>..............] - ETA: 0s - loss: 0.0724 - accuracy: 0.9775 537/938 [================>.............] - ETA: 0s - loss: 0.0736 - accuracy: 0.9774 567/938 [=================>............] - ETA: 0s - loss: 0.0739 - accuracy: 0.9772 596/938 [==================>...........] - ETA: 0s - loss: 0.0743 - accuracy: 0.9772 627/938 [===================>..........] - ETA: 0s - loss: 0.0749 - accuracy: 0.9770 657/938 [====================>.........] - ETA: 0s - loss: 0.0763 - accuracy: 0.9767 686/938 [====================>.........] - ETA: 0s - loss: 0.0771 - accuracy: 0.9764 716/938 [=====================>........] - ETA: 0s - loss: 0.0765 - accuracy: 0.9765 745/938 [======================>.......] - ETA: 0s - loss: 0.0770 - accuracy: 0.9762 775/938 [=======================>......] - ETA: 0s - loss: 0.0768 - accuracy: 0.9761 805/938 [========================>.....] - ETA: 0s - loss: 0.0770 - accuracy: 0.9763 835/938 [=========================>....] - ETA: 0s - loss: 0.0774 - accuracy: 0.9760 864/938 [==========================>...] - ETA: 0s - loss: 0.0777 - accuracy: 0.9758 894/938 [===========================>..] - ETA: 0s - loss: 0.0783 - accuracy: 0.9758 924/938 [============================>.] - ETA: 0s - loss: 0.0781 - accuracy: 0.9760 938/938 [==============================] - 2s 2ms/step - loss: 0.0783 - accuracy: 0.9760 Epoch 5/10 1/938 [..............................] - ETA: 0s - loss: 0.0508 - accuracy: 0.9844 30/938 [..............................] - ETA: 1s - loss: 0.0664 - accuracy: 0.9792 60/938 [>.............................] - ETA: 1s - loss: 0.0604 - accuracy: 0.9818 90/938 [=>............................] - ETA: 1s - loss: 0.0550 - accuracy: 0.9833 120/938 [==>...........................] - ETA: 1s - loss: 0.0531 - accuracy: 0.9835 150/938 [===>..........................] - ETA: 1s - loss: 0.0505 - accuracy: 0.9842 180/938 [====>.........................] - ETA: 1s - loss: 0.0524 - accuracy: 0.9836 210/938 [=====>........................] - ETA: 1s - loss: 0.0525 - accuracy: 0.9836 240/938 [======>.......................] - ETA: 1s - loss: 0.0530 - accuracy: 0.9832 270/938 [=======>......................] - ETA: 1s - loss: 0.0525 - accuracy: 0.9833 300/938 [========>.....................] - ETA: 1s - loss: 0.0535 - accuracy: 0.9833 330/938 [=========>....................] - ETA: 1s - loss: 0.0550 - accuracy: 0.9828 360/938 [==========>...................] - ETA: 0s - loss: 0.0567 - accuracy: 0.9820 390/938 [===========>..................] - ETA: 0s - loss: 0.0569 - accuracy: 0.9818 420/938 [============>.................] - ETA: 0s - loss: 0.0589 - accuracy: 0.9815 450/938 [=============>................] - ETA: 0s - loss: 0.0588 - accuracy: 0.9814 479/938 [==============>...............] - ETA: 0s - loss: 0.0590 - accuracy: 0.9813 508/938 [===============>..............] - ETA: 0s - loss: 0.0592 - accuracy: 0.9814 537/938 [================>.............] - ETA: 0s - loss: 0.0601 - accuracy: 0.9813 567/938 [=================>............] - ETA: 0s - loss: 0.0608 - accuracy: 0.9809 597/938 [==================>...........] - ETA: 0s - loss: 0.0610 - accuracy: 0.9809 626/938 [===================>..........] - ETA: 0s - loss: 0.0609 - accuracy: 0.9808 655/938 [===================>..........] - ETA: 0s - loss: 0.0611 - accuracy: 0.9806 684/938 [====================>.........] - ETA: 0s - loss: 0.0615 - accuracy: 0.9806 713/938 [=====================>........] - ETA: 0s - loss: 0.0614 - accuracy: 0.9805 742/938 [======================>.......] - ETA: 0s - loss: 0.0613 - accuracy: 0.9805 772/938 [=======================>......] - ETA: 0s - loss: 0.0611 - accuracy: 0.9806 800/938 [========================>.....] - ETA: 0s - loss: 0.0612 - accuracy: 0.9806 827/938 [=========================>....] - ETA: 0s - loss: 0.0611 - accuracy: 0.9807 855/938 [==========================>...] - ETA: 0s - loss: 0.0619 - accuracy: 0.9807 884/938 [===========================>..] - ETA: 0s - loss: 0.0618 - accuracy: 0.9806 914/938 [============================>.] - ETA: 0s - loss: 0.0620 - accuracy: 0.9805 938/938 [==============================] - 2s 2ms/step - loss: 0.0622 - accuracy: 0.9805 Epoch 6/10 1/938 [..............................] - ETA: 0s - loss: 0.0528 - accuracy: 0.9844 31/938 [..............................] - ETA: 1s - loss: 0.0470 - accuracy: 0.9864 61/938 [>.............................] - ETA: 1s - loss: 0.0454 - accuracy: 0.9874 91/938 [=>............................] - ETA: 1s - loss: 0.0456 - accuracy: 0.9876 121/938 [==>...........................] - ETA: 1s - loss: 0.0443 - accuracy: 0.9872 150/938 [===>..........................] - ETA: 1s - loss: 0.0442 - accuracy: 0.9872 180/938 [====>.........................] - ETA: 1s - loss: 0.0448 - accuracy: 0.9867 209/938 [=====>........................] - ETA: 1s - loss: 0.0475 - accuracy: 0.9855 238/938 [======>.......................] - ETA: 1s - loss: 0.0499 - accuracy: 0.9846 267/938 [=======>......................] - ETA: 1s - loss: 0.0496 - accuracy: 0.9848 297/938 [========>.....................] - ETA: 1s - loss: 0.0496 - accuracy: 0.9850 326/938 [=========>....................] - ETA: 1s - loss: 0.0495 - accuracy: 0.9850 356/938 [==========>...................] - ETA: 0s - loss: 0.0509 - accuracy: 0.9842 384/938 [===========>..................] - ETA: 0s - loss: 0.0507 - accuracy: 0.9845 413/938 [============>.................] - ETA: 0s - loss: 0.0498 - accuracy: 0.9847 442/938 [=============>................] - ETA: 0s - loss: 0.0498 - accuracy: 0.9846 472/938 [==============>...............] - ETA: 0s - loss: 0.0500 - accuracy: 0.9847 501/938 [===============>..............] - ETA: 0s - loss: 0.0496 - accuracy: 0.9849 531/938 [===============>..............] - ETA: 0s - loss: 0.0502 - accuracy: 0.9846 561/938 [================>.............] - ETA: 0s - loss: 0.0505 - accuracy: 0.9846 591/938 [=================>............] - ETA: 0s - loss: 0.0510 - accuracy: 0.9846 621/938 [==================>...........] - ETA: 0s - loss: 0.0513 - accuracy: 0.9843 651/938 [===================>..........] - ETA: 0s - loss: 0.0516 - accuracy: 0.9842 681/938 [====================>.........] - ETA: 0s - loss: 0.0519 - accuracy: 0.9842 710/938 [=====================>........] - ETA: 0s - loss: 0.0516 - accuracy: 0.9844 739/938 [======================>.......] - ETA: 0s - loss: 0.0515 - accuracy: 0.9844 769/938 [=======================>......] - ETA: 0s - loss: 0.0516 - accuracy: 0.9844 799/938 [========================>.....] - ETA: 0s - loss: 0.0515 - accuracy: 0.9844 829/938 [=========================>....] - ETA: 0s - loss: 0.0515 - accuracy: 0.9845 858/938 [==========================>...] - ETA: 0s - loss: 0.0515 - accuracy: 0.9844 888/938 [===========================>..] - ETA: 0s - loss: 0.0515 - accuracy: 0.9843 918/938 [============================>.] - ETA: 0s - loss: 0.0518 - accuracy: 0.9843 938/938 [==============================] - 2s 2ms/step - loss: 0.0516 - accuracy: 0.9843 Epoch 7/10 1/938 [..............................] - ETA: 0s - loss: 0.0298 - accuracy: 0.9844 31/938 [..............................] - ETA: 1s - loss: 0.0420 - accuracy: 0.9874 61/938 [>.............................] - ETA: 1s - loss: 0.0404 - accuracy: 0.9869 91/938 [=>............................] - ETA: 1s - loss: 0.0380 - accuracy: 0.9880 121/938 [==>...........................] - ETA: 1s - loss: 0.0393 - accuracy: 0.9863 151/938 [===>..........................] - ETA: 1s - loss: 0.0384 - accuracy: 0.9870 181/938 [====>.........................] - ETA: 1s - loss: 0.0380 - accuracy: 0.9873 211/938 [=====>........................] - ETA: 1s - loss: 0.0391 - accuracy: 0.9875 241/938 [======>.......................] - ETA: 1s - loss: 0.0393 - accuracy: 0.9877 271/938 [=======>......................] - ETA: 1s - loss: 0.0409 - accuracy: 0.9875 300/938 [========>.....................] - ETA: 1s - loss: 0.0413 - accuracy: 0.9874 330/938 [=========>....................] - ETA: 1s - loss: 0.0415 - accuracy: 0.9872 360/938 [==========>...................] - ETA: 0s - loss: 0.0414 - accuracy: 0.9873 389/938 [===========>..................] - ETA: 0s - loss: 0.0406 - accuracy: 0.9877 418/938 [============>.................] - ETA: 0s - loss: 0.0406 - accuracy: 0.9876 448/938 [=============>................] - ETA: 0s - loss: 0.0403 - accuracy: 0.9877 477/938 [==============>...............] - ETA: 0s - loss: 0.0406 - accuracy: 0.9876 507/938 [===============>..............] - ETA: 0s - loss: 0.0406 - accuracy: 0.9875 536/938 [================>.............] - ETA: 0s - loss: 0.0407 - accuracy: 0.9873 566/938 [=================>............] - ETA: 0s - loss: 0.0414 - accuracy: 0.9871 595/938 [==================>...........] - ETA: 0s - loss: 0.0414 - accuracy: 0.9871 624/938 [==================>...........] - ETA: 0s - loss: 0.0417 - accuracy: 0.9872 654/938 [===================>..........] - ETA: 0s - loss: 0.0426 - accuracy: 0.9868 683/938 [====================>.........] - ETA: 0s - loss: 0.0425 - accuracy: 0.9868 713/938 [=====================>........] - ETA: 0s - loss: 0.0427 - accuracy: 0.9867 743/938 [======================>.......] - ETA: 0s - loss: 0.0425 - accuracy: 0.9867 773/938 [=======================>......] - ETA: 0s - loss: 0.0426 - accuracy: 0.9866 803/938 [========================>.....] - ETA: 0s - loss: 0.0422 - accuracy: 0.9866 833/938 [=========================>....] - ETA: 0s - loss: 0.0425 - accuracy: 0.9866 863/938 [==========================>...] - ETA: 0s - loss: 0.0429 - accuracy: 0.9865 893/938 [===========================>..] - ETA: 0s - loss: 0.0430 - accuracy: 0.9864 923/938 [============================>.] - ETA: 0s - loss: 0.0432 - accuracy: 0.9863 938/938 [==============================] - 2s 2ms/step - loss: 0.0431 - accuracy: 0.9863 Epoch 8/10 1/938 [..............................] - ETA: 0s - loss: 0.2017 - accuracy: 0.9531 30/938 [..............................] - ETA: 1s - loss: 0.0369 - accuracy: 0.9885 60/938 [>.............................] - ETA: 1s - loss: 0.0368 - accuracy: 0.9885 90/938 [=>............................] - ETA: 1s - loss: 0.0344 - accuracy: 0.9894 120/938 [==>...........................] - ETA: 1s - loss: 0.0342 - accuracy: 0.9901 149/938 [===>..........................] - ETA: 1s - loss: 0.0361 - accuracy: 0.9890 179/938 [====>.........................] - ETA: 1s - loss: 0.0381 - accuracy: 0.9884 209/938 [=====>........................] - ETA: 1s - loss: 0.0392 - accuracy: 0.9877 239/938 [======>.......................] - ETA: 1s - loss: 0.0398 - accuracy: 0.9874 268/938 [=======>......................] - ETA: 1s - loss: 0.0393 - accuracy: 0.9878 298/938 [========>.....................] - ETA: 1s - loss: 0.0379 - accuracy: 0.9880 328/938 [=========>....................] - ETA: 1s - loss: 0.0374 - accuracy: 0.9884 358/938 [==========>...................] - ETA: 0s - loss: 0.0380 - accuracy: 0.9885 387/938 [===========>..................] - ETA: 0s - loss: 0.0382 - accuracy: 0.9883 416/938 [============>.................] - ETA: 0s - loss: 0.0375 - accuracy: 0.9884 446/938 [=============>................] - ETA: 0s - loss: 0.0375 - accuracy: 0.9885 476/938 [==============>...............] - ETA: 0s - loss: 0.0379 - accuracy: 0.9884 506/938 [===============>..............] - ETA: 0s - loss: 0.0383 - accuracy: 0.9880 536/938 [================>.............] - ETA: 0s - loss: 0.0388 - accuracy: 0.9877 565/938 [=================>............] - ETA: 0s - loss: 0.0397 - accuracy: 0.9875 594/938 [=================>............] - ETA: 0s - loss: 0.0399 - accuracy: 0.9873 623/938 [==================>...........] - ETA: 0s - loss: 0.0396 - accuracy: 0.9874 652/938 [===================>..........] - ETA: 0s - loss: 0.0393 - accuracy: 0.9874 682/938 [====================>.........] - ETA: 0s - loss: 0.0398 - accuracy: 0.9871 710/938 [=====================>........] - ETA: 0s - loss: 0.0400 - accuracy: 0.9870 740/938 [======================>.......] - ETA: 0s - loss: 0.0401 - accuracy: 0.9869 769/938 [=======================>......] - ETA: 0s - loss: 0.0402 - accuracy: 0.9869 799/938 [========================>.....] - ETA: 0s - loss: 0.0400 - accuracy: 0.9870 829/938 [=========================>....] - ETA: 0s - loss: 0.0401 - accuracy: 0.9870 858/938 [==========================>...] - ETA: 0s - loss: 0.0398 - accuracy: 0.9871 887/938 [===========================>..] - ETA: 0s - loss: 0.0399 - accuracy: 0.9871 916/938 [============================>.] - ETA: 0s - loss: 0.0396 - accuracy: 0.9872 938/938 [==============================] - 2s 2ms/step - loss: 0.0392 - accuracy: 0.9873 Epoch 9/10 1/938 [..............................] - ETA: 0s - loss: 0.0155 - accuracy: 1.0000 30/938 [..............................] - ETA: 1s - loss: 0.0313 - accuracy: 0.9875 59/938 [>.............................] - ETA: 1s - loss: 0.0294 - accuracy: 0.9902 89/938 [=>............................] - ETA: 1s - loss: 0.0283 - accuracy: 0.9905 120/938 [==>...........................] - ETA: 1s - loss: 0.0287 - accuracy: 0.9910 150/938 [===>..........................] - ETA: 1s - loss: 0.0307 - accuracy: 0.9903 179/938 [====>.........................] - ETA: 1s - loss: 0.0315 - accuracy: 0.9904 207/938 [=====>........................] - ETA: 1s - loss: 0.0310 - accuracy: 0.9905 234/938 [======>.......................] - ETA: 1s - loss: 0.0304 - accuracy: 0.9907 262/938 [=======>......................] - ETA: 1s - loss: 0.0313 - accuracy: 0.9900 291/938 [========>.....................] - ETA: 1s - loss: 0.0314 - accuracy: 0.9901 319/938 [=========>....................] - ETA: 1s - loss: 0.0302 - accuracy: 0.9906 348/938 [==========>...................] - ETA: 1s - loss: 0.0296 - accuracy: 0.9909 376/938 [===========>..................] - ETA: 0s - loss: 0.0303 - accuracy: 0.9906 404/938 [===========>..................] - ETA: 0s - loss: 0.0302 - accuracy: 0.9907 433/938 [============>.................] - ETA: 0s - loss: 0.0301 - accuracy: 0.9907 461/938 [=============>................] - ETA: 0s - loss: 0.0299 - accuracy: 0.9907 489/938 [==============>...............] - ETA: 0s - loss: 0.0303 - accuracy: 0.9905 518/938 [===============>..............] - ETA: 0s - loss: 0.0307 - accuracy: 0.9903 547/938 [================>.............] - ETA: 0s - loss: 0.0306 - accuracy: 0.9902 576/938 [=================>............] - ETA: 0s - loss: 0.0311 - accuracy: 0.9900 605/938 [==================>...........] - ETA: 0s - loss: 0.0316 - accuracy: 0.9899 634/938 [===================>..........] - ETA: 0s - loss: 0.0320 - accuracy: 0.9897 663/938 [====================>.........] - ETA: 0s - loss: 0.0319 - accuracy: 0.9898 693/938 [=====================>........] - ETA: 0s - loss: 0.0318 - accuracy: 0.9898 722/938 [======================>.......] - ETA: 0s - loss: 0.0319 - accuracy: 0.9897 751/938 [=======================>......] - ETA: 0s - loss: 0.0320 - accuracy: 0.9896 780/938 [=======================>......] - ETA: 0s - loss: 0.0325 - accuracy: 0.9894 809/938 [========================>.....] - ETA: 0s - loss: 0.0333 - accuracy: 0.9892 838/938 [=========================>....] - ETA: 0s - loss: 0.0330 - accuracy: 0.9893 867/938 [==========================>...] - ETA: 0s - loss: 0.0331 - accuracy: 0.9893 896/938 [===========================>..] - ETA: 0s - loss: 0.0328 - accuracy: 0.9894 925/938 [============================>.] - ETA: 0s - loss: 0.0331 - accuracy: 0.9894 938/938 [==============================] - 2s 2ms/step - loss: 0.0332 - accuracy: 0.9893 Epoch 10/10 1/938 [..............................] - ETA: 0s - loss: 0.0100 - accuracy: 1.0000 30/938 [..............................] - ETA: 1s - loss: 0.0255 - accuracy: 0.9922 59/938 [>.............................] - ETA: 1s - loss: 0.0218 - accuracy: 0.9931 89/938 [=>............................] - ETA: 1s - loss: 0.0238 - accuracy: 0.9923 117/938 [==>...........................] - ETA: 1s - loss: 0.0254 - accuracy: 0.9915 145/938 [===>..........................] - ETA: 1s - loss: 0.0273 - accuracy: 0.9912 174/938 [====>.........................] - ETA: 1s - loss: 0.0276 - accuracy: 0.9912 203/938 [=====>........................] - ETA: 1s - loss: 0.0275 - accuracy: 0.9912 232/938 [======>.......................] - ETA: 1s - loss: 0.0276 - accuracy: 0.9912 261/938 [=======>......................] - ETA: 1s - loss: 0.0276 - accuracy: 0.9910 290/938 [========>.....................] - ETA: 1s - loss: 0.0271 - accuracy: 0.9913 319/938 [=========>....................] - ETA: 1s - loss: 0.0267 - accuracy: 0.9915 347/938 [==========>...................] - ETA: 1s - loss: 0.0269 - accuracy: 0.9914 376/938 [===========>..................] - ETA: 0s - loss: 0.0265 - accuracy: 0.9917 405/938 [===========>..................] - ETA: 0s - loss: 0.0268 - accuracy: 0.9916 434/938 [============>.................] - ETA: 0s - loss: 0.0272 - accuracy: 0.9915 463/938 [=============>................] - ETA: 0s - loss: 0.0271 - accuracy: 0.9914 493/938 [==============>...............] - ETA: 0s - loss: 0.0273 - accuracy: 0.9913 522/938 [===============>..............] - ETA: 0s - loss: 0.0274 - accuracy: 0.9911 551/938 [================>.............] - ETA: 0s - loss: 0.0278 - accuracy: 0.9910 579/938 [=================>............] - ETA: 0s - loss: 0.0283 - accuracy: 0.9910 608/938 [==================>...........] - ETA: 0s - loss: 0.0287 - accuracy: 0.9908 636/938 [===================>..........] - ETA: 0s - loss: 0.0288 - accuracy: 0.9907 665/938 [====================>.........] - ETA: 0s - loss: 0.0286 - accuracy: 0.9907 694/938 [=====================>........] - ETA: 0s - loss: 0.0284 - accuracy: 0.9909 723/938 [======================>.......] - ETA: 0s - loss: 0.0284 - accuracy: 0.9909 751/938 [=======================>......] - ETA: 0s - loss: 0.0286 - accuracy: 0.9909 780/938 [=======================>......] - ETA: 0s - loss: 0.0285 - accuracy: 0.9910 809/938 [========================>.....] - ETA: 0s - loss: 0.0283 - accuracy: 0.9910 838/938 [=========================>....] - ETA: 0s - loss: 0.0283 - accuracy: 0.9910 867/938 [==========================>...] - ETA: 0s - loss: 0.0282 - accuracy: 0.9910 895/938 [===========================>..] - ETA: 0s - loss: 0.0281 - accuracy: 0.9910 924/938 [============================>.] - ETA: 0s - loss: 0.0282 - accuracy: 0.9910 938/938 [==============================] - 2s 2ms/step - loss: 0.0283 - accuracy: 0.9909
# Evaluate the network
print('Evaluate on test data:')
results = model.evaluate(x_test, y_test, batch_size=128, verbose = 2)
print('Test loss = {0}, Test acc: {1}'.format(results[0], results[1]))
Evaluate on test data: 79/79 - 0s - loss: 0.0843 - accuracy: 0.9765 Test loss = 0.08429064601659775, Test acc: 0.9764999747276306