 TensorFlow.orgで表示 |
 Google Colab で実行 |
 GitHub でソースを表示{ |
 ノートブックをダウンロード/a0} |
概要
このガイドでは、アドオンパッケージの LazyAdam オプティマイザを使用する方法を紹介します。
LazyAdam
LazyAdam は、スパースな更新をより効率的に処理する Adam オプティマイザのバリアントです。従来の Adam アルゴリズムは、トレーニング可能な変数ごとに 2 つの移動平均アキュムレータを維持します。 アキュムレータはすべてのステップで更新されます。このクラスは、スパースな変数の勾配更新をレイジーに処理します。 その時点のバッチに表示されるスパースな変数インデックスの移動平均アキュムレータのみが更新され、すべてのインデックスのアキュムレータは更新されません。アプリケーションによっては、従来の Adam オプティマイザと比べてモデルトレーニング処理能力を大幅に改善できます。 ただし、従来の Adam アルゴリズムとは若干異なるセマンティクスを提供するため、経験的結果が異なる可能性があります。
セットアップ
pip install -q -U tensorflow-addonsimport tensorflow as tf
import tensorflow_addons as tfa
# Hyperparameters
batch_size=64
epochs=10
モデルの構築
model = tf.keras.Sequential([
tf.keras.layers.Dense(64, input_shape=(784,), activation='relu', name='dense_1'),
tf.keras.layers.Dense(64, activation='relu', name='dense_2'),
tf.keras.layers.Dense(10, activation='softmax', name='predictions'),
])
データの準備
# Load MNIST dataset as NumPy arrays
dataset = {}
num_validation = 10000
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
# Preprocess the data
x_train = x_train.reshape(-1, 784).astype('float32') / 255
x_test = x_test.reshape(-1, 784).astype('float32') / 255
トレーニングと評価
一般的な keras オプティマイザを新しい tfa オプティマイザに置き換えるだけです。
# Compile the model
model.compile(
optimizer=tfa.optimizers.LazyAdam(0.001), # Utilize TFA optimizer
loss=tf.keras.losses.SparseCategoricalCrossentropy(),
metrics=['accuracy'])
# Train the network
history = model.fit(
x_train,
y_train,
batch_size=batch_size,
epochs=epochs)
Epoch 1/10 1/938 [..............................] - ETA: 10:49 - loss: 2.4328 - accuracy: 0.0625 26/938 [..............................] - ETA: 1s - loss: 2.1079 - accuracy: 0.3032 51/938 [>.............................] - ETA: 1s - loss: 1.8490 - accuracy: 0.4254 77/938 [=>............................] - ETA: 1s - loss: 1.6378 - accuracy: 0.5049 103/938 [==>...........................] - ETA: 1s - loss: 1.4801 - accuracy: 0.5593 128/938 [===>..........................] - ETA: 1s - loss: 1.3637 - accuracy: 0.5978 152/938 [===>..........................] - ETA: 1s - loss: 1.2749 - accuracy: 0.6263 177/938 [====>.........................] - ETA: 1s - loss: 1.1994 - accuracy: 0.6499 202/938 [=====>........................] - ETA: 1s - loss: 1.1364 - accuracy: 0.6693 227/938 [======>.......................] - ETA: 1s - loss: 1.0829 - accuracy: 0.6856 253/938 [=======>......................] - ETA: 1s - loss: 1.0347 - accuracy: 0.7001 279/938 [=======>......................] - ETA: 1s - loss: 0.9927 - accuracy: 0.7126 305/938 [========>.....................] - ETA: 1s - loss: 0.9557 - accuracy: 0.7237 331/938 [=========>....................] - ETA: 1s - loss: 0.9227 - accuracy: 0.7335 356/938 [==========>...................] - ETA: 1s - loss: 0.8942 - accuracy: 0.7420 381/938 [===========>..................] - ETA: 1s - loss: 0.8683 - accuracy: 0.7496 407/938 [============>.................] - ETA: 1s - loss: 0.8439 - accuracy: 0.7569 433/938 [============>.................] - ETA: 1s - loss: 0.8216 - accuracy: 0.7635 459/938 [=============>................] - ETA: 0s - loss: 0.8010 - accuracy: 0.7695 484/938 [==============>...............] - ETA: 0s - loss: 0.7828 - accuracy: 0.7749 510/938 [===============>..............] - ETA: 0s - loss: 0.7651 - accuracy: 0.7801 534/938 [================>.............] - ETA: 0s - loss: 0.7498 - accuracy: 0.7845 558/938 [================>.............] - ETA: 0s - loss: 0.7354 - accuracy: 0.7887 584/938 [=================>............] - ETA: 0s - loss: 0.7208 - accuracy: 0.7930 609/938 [==================>...........] - ETA: 0s - loss: 0.7075 - accuracy: 0.7969 634/938 [===================>..........] - ETA: 0s - loss: 0.6951 - accuracy: 0.8005 659/938 [====================>.........] - ETA: 0s - loss: 0.6833 - accuracy: 0.8039 684/938 [====================>.........] - ETA: 0s - loss: 0.6721 - accuracy: 0.8072 710/938 [=====================>........] - ETA: 0s - loss: 0.6610 - accuracy: 0.8104 736/938 [======================>.......] - ETA: 0s - loss: 0.6506 - accuracy: 0.8134 762/938 [=======================>......] - ETA: 0s - loss: 0.6407 - accuracy: 0.8163 789/938 [========================>.....] - ETA: 0s - loss: 0.6308 - accuracy: 0.8191 815/938 [=========================>....] - ETA: 0s - loss: 0.6218 - accuracy: 0.8217 841/938 [=========================>....] - ETA: 0s - loss: 0.6132 - accuracy: 0.8242 867/938 [==========================>...] - ETA: 0s - loss: 0.6049 - accuracy: 0.8266 893/938 [===========================>..] - ETA: 0s - loss: 0.5970 - accuracy: 0.8288 919/938 [============================>.] - ETA: 0s - loss: 0.5895 - accuracy: 0.8310 938/938 [==============================] - 3s 2ms/step - loss: 0.5839 - accuracy: 0.8326 Epoch 2/10 1/938 [..............................] - ETA: 1s - loss: 0.1893 - accuracy: 0.9375 28/938 [..............................] - ETA: 1s - loss: 0.1563 - accuracy: 0.9530 54/938 [>.............................] - ETA: 1s - loss: 0.1601 - accuracy: 0.9535 79/938 [=>............................] - ETA: 1s - loss: 0.1608 - accuracy: 0.9540 105/938 [==>...........................] - ETA: 1s - loss: 0.1621 - accuracy: 0.9542 131/938 [===>..........................] - ETA: 1s - loss: 0.1634 - accuracy: 0.9540 156/938 [===>..........................] - ETA: 1s - loss: 0.1631 - accuracy: 0.9542 182/938 [====>.........................] - ETA: 1s - loss: 0.1625 - accuracy: 0.9543 208/938 [=====>........................] - ETA: 1s - loss: 0.1618 - accuracy: 0.9545 235/938 [======>.......................] - ETA: 1s - loss: 0.1610 - accuracy: 0.9548 262/938 [=======>......................] - ETA: 1s - loss: 0.1603 - accuracy: 0.9549 290/938 [========>.....................] - ETA: 1s - loss: 0.1596 - accuracy: 0.9551 318/938 [=========>....................] - ETA: 1s - loss: 0.1590 - accuracy: 0.9553 345/938 [==========>...................] - ETA: 1s - loss: 0.1584 - accuracy: 0.9554 373/938 [==========>...................] - ETA: 1s - loss: 0.1578 - accuracy: 0.9555 400/938 [===========>..................] - ETA: 1s - loss: 0.1572 - accuracy: 0.9556 426/938 [============>.................] - ETA: 0s - loss: 0.1567 - accuracy: 0.9557 452/938 [=============>................] - ETA: 0s - loss: 0.1563 - accuracy: 0.9557 479/938 [==============>...............] - ETA: 0s - loss: 0.1558 - accuracy: 0.9558 506/938 [===============>..............] - ETA: 0s - loss: 0.1554 - accuracy: 0.9559 531/938 [===============>..............] - ETA: 0s - loss: 0.1550 - accuracy: 0.9559 557/938 [================>.............] - ETA: 0s - loss: 0.1547 - accuracy: 0.9559 583/938 [=================>............] - ETA: 0s - loss: 0.1544 - accuracy: 0.9560 609/938 [==================>...........] - ETA: 0s - loss: 0.1541 - accuracy: 0.9560 635/938 [===================>..........] - ETA: 0s - loss: 0.1539 - accuracy: 0.9560 661/938 [====================>.........] - ETA: 0s - loss: 0.1536 - accuracy: 0.9560 687/938 [====================>.........] - ETA: 0s - loss: 0.1534 - accuracy: 0.9560 714/938 [=====================>........] - ETA: 0s - loss: 0.1531 - accuracy: 0.9561 741/938 [======================>.......] - ETA: 0s - loss: 0.1528 - accuracy: 0.9561 767/938 [=======================>......] - ETA: 0s - loss: 0.1526 - accuracy: 0.9561 793/938 [========================>.....] - ETA: 0s - loss: 0.1523 - accuracy: 0.9562 819/938 [=========================>....] - ETA: 0s - loss: 0.1520 - accuracy: 0.9562 847/938 [==========================>...] - ETA: 0s - loss: 0.1517 - accuracy: 0.9562 875/938 [==========================>...] - ETA: 0s - loss: 0.1515 - accuracy: 0.9563 903/938 [===========================>..] - ETA: 0s - loss: 0.1512 - accuracy: 0.9563 930/938 [============================>.] - ETA: 0s - loss: 0.1509 - accuracy: 0.9564 938/938 [==============================] - 2s 2ms/step - loss: 0.1508 - accuracy: 0.9564 Epoch 3/10 1/938 [..............................] - ETA: 1s - loss: 0.1642 - accuracy: 0.9375 27/938 [..............................] - ETA: 1s - loss: 0.1110 - accuracy: 0.9620 53/938 [>.............................] - ETA: 1s - loss: 0.1025 - accuracy: 0.9657 80/938 [=>............................] - ETA: 1s - loss: 0.1015 - accuracy: 0.9665 106/938 [==>...........................] - ETA: 1s - loss: 0.1024 - accuracy: 0.9670 132/938 [===>..........................] - ETA: 1s - loss: 0.1035 - accuracy: 0.9672 158/938 [====>.........................] - ETA: 1s - loss: 0.1048 - accuracy: 0.9672 185/938 [====>.........................] - ETA: 1s - loss: 0.1055 - accuracy: 0.9674 212/938 [=====>........................] - ETA: 1s - loss: 0.1059 - accuracy: 0.9675 238/938 [======>.......................] - ETA: 1s - loss: 0.1061 - accuracy: 0.9676 265/938 [=======>......................] - ETA: 1s - loss: 0.1060 - accuracy: 0.9678 292/938 [========>.....................] - ETA: 1s - loss: 0.1059 - accuracy: 0.9680 319/938 [=========>....................] - ETA: 1s - loss: 0.1057 - accuracy: 0.9681 346/938 [==========>...................] - ETA: 1s - loss: 0.1056 - accuracy: 0.9683 373/938 [==========>...................] - ETA: 1s - loss: 0.1056 - accuracy: 0.9683 400/938 [===========>..................] - ETA: 1s - loss: 0.1057 - accuracy: 0.9684 427/938 [============>.................] - ETA: 0s - loss: 0.1057 - accuracy: 0.9684 454/938 [=============>................] - ETA: 0s - loss: 0.1059 - accuracy: 0.9684 481/938 [==============>...............] - ETA: 0s - loss: 0.1059 - accuracy: 0.9684 507/938 [===============>..............] - ETA: 0s - loss: 0.1060 - accuracy: 0.9684 533/938 [================>.............] - ETA: 0s - loss: 0.1059 - accuracy: 0.9684 559/938 [================>.............] - ETA: 0s - loss: 0.1059 - accuracy: 0.9685 584/938 [=================>............] - ETA: 0s - loss: 0.1058 - accuracy: 0.9685 610/938 [==================>...........] - ETA: 0s - loss: 0.1057 - accuracy: 0.9685 636/938 [===================>..........] - ETA: 0s - loss: 0.1057 - accuracy: 0.9686 663/938 [====================>.........] - ETA: 0s - loss: 0.1056 - accuracy: 0.9686 689/938 [=====================>........] - ETA: 0s - loss: 0.1056 - accuracy: 0.9686 716/938 [=====================>........] - ETA: 0s - loss: 0.1056 - accuracy: 0.9686 743/938 [======================>.......] - ETA: 0s - loss: 0.1055 - accuracy: 0.9686 770/938 [=======================>......] - ETA: 0s - loss: 0.1055 - accuracy: 0.9686 796/938 [========================>.....] - ETA: 0s - loss: 0.1054 - accuracy: 0.9686 823/938 [=========================>....] - ETA: 0s - loss: 0.1054 - accuracy: 0.9686 849/938 [==========================>...] - ETA: 0s - loss: 0.1054 - accuracy: 0.9686 876/938 [===========================>..] - ETA: 0s - loss: 0.1054 - accuracy: 0.9686 903/938 [===========================>..] - ETA: 0s - loss: 0.1054 - accuracy: 0.9686 930/938 [============================>.] - ETA: 0s - loss: 0.1053 - accuracy: 0.9686 938/938 [==============================] - 2s 2ms/step - loss: 0.1053 - accuracy: 0.9686 Epoch 4/10 1/938 [..............................] - ETA: 1s - loss: 0.0866 - accuracy: 0.9844 29/938 [..............................] - ETA: 1s - loss: 0.0788 - accuracy: 0.9754 56/938 [>.............................] - ETA: 1s - loss: 0.0808 - accuracy: 0.9750 83/938 [=>............................] - ETA: 1s - loss: 0.0835 - accuracy: 0.9743 108/938 [==>...........................] - ETA: 1s - loss: 0.0843 - accuracy: 0.9742 134/938 [===>..........................] - ETA: 1s - loss: 0.0853 - accuracy: 0.9740 161/938 [====>.........................] - ETA: 1s - loss: 0.0856 - accuracy: 0.9740 187/938 [====>.........................] - ETA: 1s - loss: 0.0858 - accuracy: 0.9740 214/938 [=====>........................] - ETA: 1s - loss: 0.0857 - accuracy: 0.9741 240/938 [======>.......................] - ETA: 1s - loss: 0.0855 - accuracy: 0.9742 267/938 [=======>......................] - ETA: 1s - loss: 0.0854 - accuracy: 0.9743 295/938 [========>.....................] - ETA: 1s - loss: 0.0851 - accuracy: 0.9744 322/938 [=========>....................] - ETA: 1s - loss: 0.0848 - accuracy: 0.9745 347/938 [==========>...................] - ETA: 1s - loss: 0.0845 - accuracy: 0.9746 374/938 [==========>...................] - ETA: 1s - loss: 0.0844 - accuracy: 0.9746 402/938 [===========>..................] - ETA: 1s - loss: 0.0843 - accuracy: 0.9746 429/938 [============>.................] - ETA: 0s - loss: 0.0843 - accuracy: 0.9746 455/938 [=============>................] - ETA: 0s - loss: 0.0843 - accuracy: 0.9746 482/938 [==============>...............] - ETA: 0s - loss: 0.0842 - accuracy: 0.9746 509/938 [===============>..............] - ETA: 0s - loss: 0.0841 - accuracy: 0.9746 536/938 [================>.............] - ETA: 0s - loss: 0.0840 - accuracy: 0.9746 563/938 [=================>............] - ETA: 0s - loss: 0.0839 - accuracy: 0.9746 590/938 [=================>............] - ETA: 0s - loss: 0.0838 - accuracy: 0.9747 617/938 [==================>...........] - ETA: 0s - loss: 0.0838 - accuracy: 0.9747 644/938 [===================>..........] - ETA: 0s - loss: 0.0837 - accuracy: 0.9747 671/938 [====================>.........] - ETA: 0s - loss: 0.0836 - accuracy: 0.9747 696/938 [=====================>........] - ETA: 0s - loss: 0.0836 - accuracy: 0.9747 723/938 [======================>.......] - ETA: 0s - loss: 0.0836 - accuracy: 0.9747 750/938 [======================>.......] - ETA: 0s - loss: 0.0835 - accuracy: 0.9748 778/938 [=======================>......] - ETA: 0s - loss: 0.0835 - accuracy: 0.9748 806/938 [========================>.....] - ETA: 0s - loss: 0.0835 - accuracy: 0.9748 833/938 [=========================>....] - ETA: 0s - loss: 0.0835 - accuracy: 0.9748 861/938 [==========================>...] - ETA: 0s - loss: 0.0834 - accuracy: 0.9748 888/938 [===========================>..] - ETA: 0s - loss: 0.0834 - accuracy: 0.9748 916/938 [============================>.] - ETA: 0s - loss: 0.0834 - accuracy: 0.9748 938/938 [==============================] - 2s 2ms/step - loss: 0.0833 - accuracy: 0.9748 Epoch 5/10 1/938 [..............................] - ETA: 1s - loss: 0.0616 - accuracy: 0.9844 27/938 [..............................] - ETA: 1s - loss: 0.0572 - accuracy: 0.9816 54/938 [>.............................] - ETA: 1s - loss: 0.0590 - accuracy: 0.9807 80/938 [=>............................] - ETA: 1s - loss: 0.0621 - accuracy: 0.9801 107/938 [==>...........................] - ETA: 1s - loss: 0.0641 - accuracy: 0.9798 135/938 [===>..........................] - ETA: 1s - loss: 0.0651 - accuracy: 0.9797 163/938 [====>.........................] - ETA: 1s - loss: 0.0652 - accuracy: 0.9797 191/938 [=====>........................] - ETA: 1s - loss: 0.0651 - accuracy: 0.9798 219/938 [======>.......................] - ETA: 1s - loss: 0.0649 - accuracy: 0.9799 245/938 [======>.......................] - ETA: 1s - loss: 0.0647 - accuracy: 0.9800 272/938 [=======>......................] - ETA: 1s - loss: 0.0646 - accuracy: 0.9801 299/938 [========>.....................] - ETA: 1s - loss: 0.0644 - accuracy: 0.9802 326/938 [=========>....................] - ETA: 1s - loss: 0.0643 - accuracy: 0.9802 353/938 [==========>...................] - ETA: 1s - loss: 0.0642 - accuracy: 0.9803 380/938 [===========>..................] - ETA: 1s - loss: 0.0641 - accuracy: 0.9803 408/938 [============>.................] - ETA: 0s - loss: 0.0641 - accuracy: 0.9803 436/938 [============>.................] - ETA: 0s - loss: 0.0641 - accuracy: 0.9803 463/938 [=============>................] - ETA: 0s - loss: 0.0641 - accuracy: 0.9803 490/938 [==============>...............] - ETA: 0s - loss: 0.0641 - accuracy: 0.9803 517/938 [===============>..............] - ETA: 0s - loss: 0.0641 - accuracy: 0.9803 544/938 [================>.............] - ETA: 0s - loss: 0.0641 - accuracy: 0.9803 571/938 [=================>............] - ETA: 0s - loss: 0.0642 - accuracy: 0.9803 598/938 [==================>...........] - ETA: 0s - loss: 0.0642 - accuracy: 0.9802 625/938 [==================>...........] - ETA: 0s - loss: 0.0643 - accuracy: 0.9802 652/938 [===================>..........] - ETA: 0s - loss: 0.0644 - accuracy: 0.9802 679/938 [====================>.........] - ETA: 0s - loss: 0.0645 - accuracy: 0.9801 706/938 [=====================>........] - ETA: 0s - loss: 0.0646 - accuracy: 0.9801 733/938 [======================>.......] - ETA: 0s - loss: 0.0647 - accuracy: 0.9801 761/938 [=======================>......] - ETA: 0s - loss: 0.0648 - accuracy: 0.9800 787/938 [========================>.....] - ETA: 0s - loss: 0.0649 - accuracy: 0.9800 813/938 [=========================>....] - ETA: 0s - loss: 0.0650 - accuracy: 0.9799 839/938 [=========================>....] - ETA: 0s - loss: 0.0651 - accuracy: 0.9799 867/938 [==========================>...] - ETA: 0s - loss: 0.0653 - accuracy: 0.9799 894/938 [===========================>..] - ETA: 0s - loss: 0.0654 - accuracy: 0.9798 921/938 [============================>.] - ETA: 0s - loss: 0.0654 - accuracy: 0.9798 938/938 [==============================] - 2s 2ms/step - loss: 0.0655 - accuracy: 0.9798 Epoch 6/10 1/938 [..............................] - ETA: 1s - loss: 0.0200 - accuracy: 1.0000 28/938 [..............................] - ETA: 1s - loss: 0.0493 - accuracy: 0.9804 55/938 [>.............................] - ETA: 1s - loss: 0.0496 - accuracy: 0.9813 82/938 [=>............................] - ETA: 1s - loss: 0.0500 - accuracy: 0.9820 110/938 [==>...........................] - ETA: 1s - loss: 0.0507 - accuracy: 0.9821 138/938 [===>..........................] - ETA: 1s - loss: 0.0511 - accuracy: 0.9824 165/938 [====>.........................] - ETA: 1s - loss: 0.0509 - accuracy: 0.9827 192/938 [=====>........................] - ETA: 1s - loss: 0.0507 - accuracy: 0.9830 219/938 [======>.......................] - ETA: 1s - loss: 0.0505 - accuracy: 0.9832 246/938 [======>.......................] - ETA: 1s - loss: 0.0505 - accuracy: 0.9833 274/938 [=======>......................] - ETA: 1s - loss: 0.0506 - accuracy: 0.9833 301/938 [========>.....................] - ETA: 1s - loss: 0.0508 - accuracy: 0.9833 328/938 [=========>....................] - ETA: 1s - loss: 0.0508 - accuracy: 0.9833 355/938 [==========>...................] - ETA: 1s - loss: 0.0509 - accuracy: 0.9834 381/938 [===========>..................] - ETA: 1s - loss: 0.0510 - accuracy: 0.9834 407/938 [============>.................] - ETA: 0s - loss: 0.0511 - accuracy: 0.9834 434/938 [============>.................] - ETA: 0s - loss: 0.0511 - accuracy: 0.9835 460/938 [=============>................] - ETA: 0s - loss: 0.0512 - accuracy: 0.9835 488/938 [==============>...............] - ETA: 0s - loss: 0.0513 - accuracy: 0.9835 516/938 [===============>..............] - ETA: 0s - loss: 0.0514 - accuracy: 0.9835 543/938 [================>.............] - ETA: 0s - loss: 0.0515 - accuracy: 0.9835 571/938 [=================>............] - ETA: 0s - loss: 0.0515 - accuracy: 0.9836 599/938 [==================>...........] - ETA: 0s - loss: 0.0515 - accuracy: 0.9836 627/938 [===================>..........] - ETA: 0s - loss: 0.0515 - accuracy: 0.9836 654/938 [===================>..........] - ETA: 0s - loss: 0.0516 - accuracy: 0.9836 681/938 [====================>.........] - ETA: 0s - loss: 0.0516 - accuracy: 0.9836 707/938 [=====================>........] - ETA: 0s - loss: 0.0517 - accuracy: 0.9836 734/938 [======================>.......] - ETA: 0s - loss: 0.0518 - accuracy: 0.9836 760/938 [=======================>......] - ETA: 0s - loss: 0.0519 - accuracy: 0.9836 786/938 [========================>.....] - ETA: 0s - loss: 0.0520 - accuracy: 0.9836 813/938 [=========================>....] - ETA: 0s - loss: 0.0521 - accuracy: 0.9836 840/938 [=========================>....] - ETA: 0s - loss: 0.0522 - accuracy: 0.9835 866/938 [==========================>...] - ETA: 0s - loss: 0.0523 - accuracy: 0.9835 892/938 [===========================>..] - ETA: 0s - loss: 0.0524 - accuracy: 0.9835 919/938 [============================>.] - ETA: 0s - loss: 0.0525 - accuracy: 0.9835 938/938 [==============================] - 2s 2ms/step - loss: 0.0525 - accuracy: 0.9835 Epoch 7/10 1/938 [..............................] - ETA: 1s - loss: 0.0407 - accuracy: 0.9844 28/938 [..............................] - ETA: 1s - loss: 0.0473 - accuracy: 0.9839 55/938 [>.............................] - ETA: 1s - loss: 0.0486 - accuracy: 0.9835 83/938 [=>............................] - ETA: 1s - loss: 0.0485 - accuracy: 0.9837 111/938 [==>...........................] - ETA: 1s - loss: 0.0495 - accuracy: 0.9833 137/938 [===>..........................] - ETA: 1s - loss: 0.0499 - accuracy: 0.9832 164/938 [====>.........................] - ETA: 1s - loss: 0.0497 - accuracy: 0.9832 191/938 [=====>........................] - ETA: 1s - loss: 0.0493 - accuracy: 0.9833 219/938 [======>.......................] - ETA: 1s - loss: 0.0487 - accuracy: 0.9835 247/938 [======>.......................] - ETA: 1s - loss: 0.0482 - accuracy: 0.9838 274/938 [=======>......................] - ETA: 1s - loss: 0.0477 - accuracy: 0.9839 299/938 [========>.....................] - ETA: 1s - loss: 0.0474 - accuracy: 0.9841 325/938 [=========>....................] - ETA: 1s - loss: 0.0471 - accuracy: 0.9842 351/938 [==========>...................] - ETA: 1s - loss: 0.0469 - accuracy: 0.9844 377/938 [===========>..................] - ETA: 1s - loss: 0.0466 - accuracy: 0.9845 404/938 [===========>..................] - ETA: 1s - loss: 0.0464 - accuracy: 0.9846 431/938 [============>.................] - ETA: 0s - loss: 0.0462 - accuracy: 0.9847 458/938 [=============>................] - ETA: 0s - loss: 0.0460 - accuracy: 0.9848 484/938 [==============>...............] - ETA: 0s - loss: 0.0459 - accuracy: 0.9849 511/938 [===============>..............] - ETA: 0s - loss: 0.0458 - accuracy: 0.9849 538/938 [================>.............] - ETA: 0s - loss: 0.0457 - accuracy: 0.9850 565/938 [=================>............] - ETA: 0s - loss: 0.0456 - accuracy: 0.9850 592/938 [=================>............] - ETA: 0s - loss: 0.0455 - accuracy: 0.9851 620/938 [==================>...........] - ETA: 0s - loss: 0.0455 - accuracy: 0.9851 648/938 [===================>..........] - ETA: 0s - loss: 0.0454 - accuracy: 0.9852 676/938 [====================>.........] - ETA: 0s - loss: 0.0454 - accuracy: 0.9852 703/938 [=====================>........] - ETA: 0s - loss: 0.0454 - accuracy: 0.9852 731/938 [======================>.......] - ETA: 0s - loss: 0.0453 - accuracy: 0.9852 758/938 [=======================>......] - ETA: 0s - loss: 0.0453 - accuracy: 0.9853 785/938 [========================>.....] - ETA: 0s - loss: 0.0454 - accuracy: 0.9853 812/938 [========================>.....] - ETA: 0s - loss: 0.0454 - accuracy: 0.9853 839/938 [=========================>....] - ETA: 0s - loss: 0.0454 - accuracy: 0.9853 865/938 [==========================>...] - ETA: 0s - loss: 0.0454 - accuracy: 0.9853 891/938 [===========================>..] - ETA: 0s - loss: 0.0455 - accuracy: 0.9853 918/938 [============================>.] - ETA: 0s - loss: 0.0455 - accuracy: 0.9853 938/938 [==============================] - 2s 2ms/step - loss: 0.0455 - accuracy: 0.9853 Epoch 8/10 1/938 [..............................] - ETA: 1s - loss: 0.0322 - accuracy: 1.0000 28/938 [..............................] - ETA: 1s - loss: 0.0313 - accuracy: 0.9933 55/938 [>.............................] - ETA: 1s - loss: 0.0323 - accuracy: 0.9923 82/938 [=>............................] - ETA: 1s - loss: 0.0328 - accuracy: 0.9919 109/938 [==>...........................] - ETA: 1s - loss: 0.0330 - accuracy: 0.9915 136/938 [===>..........................] - ETA: 1s - loss: 0.0333 - accuracy: 0.9912 162/938 [====>.........................] - ETA: 1s - loss: 0.0333 - accuracy: 0.9911 189/938 [=====>........................] - ETA: 1s - loss: 0.0331 - accuracy: 0.9911 217/938 [=====>........................] - ETA: 1s - loss: 0.0331 - accuracy: 0.9910 245/938 [======>.......................] - ETA: 1s - loss: 0.0333 - accuracy: 0.9909 273/938 [=======>......................] - ETA: 1s - loss: 0.0334 - accuracy: 0.9907 300/938 [========>.....................] - ETA: 1s - loss: 0.0336 - accuracy: 0.9906 328/938 [=========>....................] - ETA: 1s - loss: 0.0338 - accuracy: 0.9905 355/938 [==========>...................] - ETA: 1s - loss: 0.0340 - accuracy: 0.9903 382/938 [===========>..................] - ETA: 1s - loss: 0.0341 - accuracy: 0.9903 408/938 [============>.................] - ETA: 0s - loss: 0.0342 - accuracy: 0.9902 435/938 [============>.................] - ETA: 0s - loss: 0.0343 - accuracy: 0.9901 463/938 [=============>................] - ETA: 0s - loss: 0.0344 - accuracy: 0.9900 490/938 [==============>...............] - ETA: 0s - loss: 0.0345 - accuracy: 0.9899 518/938 [===============>..............] - ETA: 0s - loss: 0.0346 - accuracy: 0.9899 546/938 [================>.............] - ETA: 0s - loss: 0.0347 - accuracy: 0.9898 572/938 [=================>............] - ETA: 0s - loss: 0.0347 - accuracy: 0.9898 598/938 [==================>...........] - ETA: 0s - loss: 0.0348 - accuracy: 0.9897 624/938 [==================>...........] - ETA: 0s - loss: 0.0349 - accuracy: 0.9897 650/938 [===================>..........] - ETA: 0s - loss: 0.0350 - accuracy: 0.9896 675/938 [====================>.........] - ETA: 0s - loss: 0.0350 - accuracy: 0.9896 700/938 [=====================>........] - ETA: 0s - loss: 0.0351 - accuracy: 0.9895 725/938 [======================>.......] - ETA: 0s - loss: 0.0351 - accuracy: 0.9895 749/938 [======================>.......] - ETA: 0s - loss: 0.0352 - accuracy: 0.9894 774/938 [=======================>......] - ETA: 0s - loss: 0.0353 - accuracy: 0.9894 799/938 [========================>.....] - ETA: 0s - loss: 0.0354 - accuracy: 0.9894 825/938 [=========================>....] - ETA: 0s - loss: 0.0355 - accuracy: 0.9893 851/938 [==========================>...] - ETA: 0s - loss: 0.0357 - accuracy: 0.9893 878/938 [===========================>..] - ETA: 0s - loss: 0.0358 - accuracy: 0.9892 904/938 [===========================>..] - ETA: 0s - loss: 0.0359 - accuracy: 0.9891 931/938 [============================>.] - ETA: 0s - loss: 0.0360 - accuracy: 0.9891 938/938 [==============================] - 2s 2ms/step - loss: 0.0360 - accuracy: 0.9891 Epoch 9/10 1/938 [..............................] - ETA: 1s - loss: 0.0282 - accuracy: 0.9844 29/938 [..............................] - ETA: 1s - loss: 0.0312 - accuracy: 0.9905 56/938 [>.............................] - ETA: 1s - loss: 0.0333 - accuracy: 0.9899 81/938 [=>............................] - ETA: 1s - loss: 0.0330 - accuracy: 0.9901 108/938 [==>...........................] - ETA: 1s - loss: 0.0328 - accuracy: 0.9903 136/938 [===>..........................] - ETA: 1s - loss: 0.0324 - accuracy: 0.9905 163/938 [====>.........................] - ETA: 1s - loss: 0.0321 - accuracy: 0.9905 189/938 [=====>........................] - ETA: 1s - loss: 0.0319 - accuracy: 0.9905 215/938 [=====>........................] - ETA: 1s - loss: 0.0319 - accuracy: 0.9905 241/938 [======>.......................] - ETA: 1s - loss: 0.0319 - accuracy: 0.9904 266/938 [=======>......................] - ETA: 1s - loss: 0.0319 - accuracy: 0.9903 292/938 [========>.....................] - ETA: 1s - loss: 0.0320 - accuracy: 0.9903 318/938 [=========>....................] - ETA: 1s - loss: 0.0321 - accuracy: 0.9902 344/938 [==========>...................] - ETA: 1s - loss: 0.0322 - accuracy: 0.9901 370/938 [==========>...................] - ETA: 1s - loss: 0.0324 - accuracy: 0.9900 396/938 [===========>..................] - ETA: 1s - loss: 0.0325 - accuracy: 0.9900 422/938 [============>.................] - ETA: 0s - loss: 0.0326 - accuracy: 0.9899 447/938 [=============>................] - ETA: 0s - loss: 0.0326 - accuracy: 0.9899 473/938 [==============>...............] - ETA: 0s - loss: 0.0326 - accuracy: 0.9899 500/938 [==============>...............] - ETA: 0s - loss: 0.0326 - accuracy: 0.9899 526/938 [===============>..............] - ETA: 0s - loss: 0.0327 - accuracy: 0.9898 552/938 [================>.............] - ETA: 0s - loss: 0.0327 - accuracy: 0.9898 579/938 [=================>............] - ETA: 0s - loss: 0.0328 - accuracy: 0.9897 605/938 [==================>...........] - ETA: 0s - loss: 0.0329 - accuracy: 0.9897 632/938 [===================>..........] - ETA: 0s - loss: 0.0329 - accuracy: 0.9897 658/938 [====================>.........] - ETA: 0s - loss: 0.0330 - accuracy: 0.9896 684/938 [====================>.........] - ETA: 0s - loss: 0.0330 - accuracy: 0.9896 710/938 [=====================>........] - ETA: 0s - loss: 0.0331 - accuracy: 0.9896 735/938 [======================>.......] - ETA: 0s - loss: 0.0332 - accuracy: 0.9895 759/938 [=======================>......] - ETA: 0s - loss: 0.0333 - accuracy: 0.9895 784/938 [========================>.....] - ETA: 0s - loss: 0.0333 - accuracy: 0.9895 808/938 [========================>.....] - ETA: 0s - loss: 0.0334 - accuracy: 0.9894 833/938 [=========================>....] - ETA: 0s - loss: 0.0335 - accuracy: 0.9894 859/938 [==========================>...] - ETA: 0s - loss: 0.0336 - accuracy: 0.9893 884/938 [===========================>..] - ETA: 0s - loss: 0.0337 - accuracy: 0.9893 910/938 [============================>.] - ETA: 0s - loss: 0.0338 - accuracy: 0.9893 936/938 [============================>.] - ETA: 0s - loss: 0.0339 - accuracy: 0.9892 938/938 [==============================] - 2s 2ms/step - loss: 0.0339 - accuracy: 0.9892 Epoch 10/10 1/938 [..............................] - ETA: 2s - loss: 0.0568 - accuracy: 0.9844 28/938 [..............................] - ETA: 1s - loss: 0.0269 - accuracy: 0.9928 53/938 [>.............................] - ETA: 1s - loss: 0.0229 - accuracy: 0.9940 78/938 [=>............................] - ETA: 1s - loss: 0.0219 - accuracy: 0.9942 104/938 [==>...........................] - ETA: 1s - loss: 0.0223 - accuracy: 0.9939 130/938 [===>..........................] - ETA: 1s - loss: 0.0229 - accuracy: 0.9936 155/938 [===>..........................] - ETA: 1s - loss: 0.0236 - accuracy: 0.9934 182/938 [====>.........................] - ETA: 1s - loss: 0.0241 - accuracy: 0.9931 207/938 [=====>........................] - ETA: 1s - loss: 0.0244 - accuracy: 0.9929 232/938 [======>.......................] - ETA: 1s - loss: 0.0248 - accuracy: 0.9928 257/938 [=======>......................] - ETA: 1s - loss: 0.0251 - accuracy: 0.9926 283/938 [========>.....................] - ETA: 1s - loss: 0.0253 - accuracy: 0.9925 309/938 [========>.....................] - ETA: 1s - loss: 0.0255 - accuracy: 0.9924 335/938 [=========>....................] - ETA: 1s - loss: 0.0256 - accuracy: 0.9923 361/938 [==========>...................] - ETA: 1s - loss: 0.0258 - accuracy: 0.9923 388/938 [===========>..................] - ETA: 1s - loss: 0.0259 - accuracy: 0.9922 414/938 [============>.................] - ETA: 1s - loss: 0.0260 - accuracy: 0.9921 440/938 [=============>................] - ETA: 0s - loss: 0.0261 - accuracy: 0.9921 467/938 [=============>................] - ETA: 0s - loss: 0.0262 - accuracy: 0.9920 494/938 [==============>...............] - ETA: 0s - loss: 0.0263 - accuracy: 0.9920 522/938 [===============>..............] - ETA: 0s - loss: 0.0264 - accuracy: 0.9920 549/938 [================>.............] - ETA: 0s - loss: 0.0264 - accuracy: 0.9919 575/938 [=================>............] - ETA: 0s - loss: 0.0265 - accuracy: 0.9919 602/938 [==================>...........] - ETA: 0s - loss: 0.0265 - accuracy: 0.9919 628/938 [===================>..........] - ETA: 0s - loss: 0.0266 - accuracy: 0.9919 653/938 [===================>..........] - ETA: 0s - loss: 0.0266 - accuracy: 0.9919 678/938 [====================>.........] - ETA: 0s - loss: 0.0266 - accuracy: 0.9919 704/938 [=====================>........] - ETA: 0s - loss: 0.0266 - accuracy: 0.9919 730/938 [======================>.......] - ETA: 0s - loss: 0.0267 - accuracy: 0.9918 756/938 [=======================>......] - ETA: 0s - loss: 0.0267 - accuracy: 0.9918 782/938 [========================>.....] - ETA: 0s - loss: 0.0267 - accuracy: 0.9918 809/938 [========================>.....] - ETA: 0s - loss: 0.0268 - accuracy: 0.9918 836/938 [=========================>....] - ETA: 0s - loss: 0.0268 - accuracy: 0.9918 862/938 [==========================>...] - ETA: 0s - loss: 0.0269 - accuracy: 0.9917 889/938 [===========================>..] - ETA: 0s - loss: 0.0269 - accuracy: 0.9917 917/938 [============================>.] - ETA: 0s - loss: 0.0270 - accuracy: 0.9917 938/938 [==============================] - 2s 2ms/step - loss: 0.0270 - accuracy: 0.9917
# Evaluate the network
print('Evaluate on test data:')
results = model.evaluate(x_test, y_test, batch_size=128, verbose = 2)
print('Test loss = {0}, Test acc: {1}'.format(results[0], results[1]))
Evaluate on test data: 79/79 - 0s - loss: 0.0875 - accuracy: 0.9773 Test loss = 0.08750879764556885, Test acc: 0.9772999882698059