 TensorFlow.org에서 보기 |
 Google Colab에서 실행하기 |
 GitHub에서 소스 보기 |
 노트북 다운로드하기 |
개요
이 노트북은 애드온 패키지에서 게으른 adam 옵티마이저를 사용하는 방법을 보여줍니다.
LazyAdam
LazyAdam은 희소 업데이트를 보다 효율적으로 처리하는 Adam 옵티마이저의 변형입니다. 원래 Adam 알고리즘은 각 훈련 가능한 변수에 대해 두 개의 이동 평균 누산기(moving-average accumulator)를 유지합니다. 누산기는 모든 단계에서 업데이트됩니다. 이 클래스는 희소 변수에 대한 그래디언트 업데이트의 지연 처리를 제공합니다. 모든 인덱스에 대한 누산기를 업데이트하지 않고 현재 배치에 나타나는 희소 변수 인덱스에 대한 이동 평균 누산기만 업데이트합니다. 원래 Adam 옵티마이저와 비교하여 일부 애플리케이션의 모델 훈련 처리량을 크게 향상할 수 있습니다. 그러나 원래 Adam 알고리즘과 약간 다른 의미를 제공하며 경험적 결과가 다를 수 있습니다.
설정
pip install -q -U tensorflow-addonsimport tensorflow as tf
import tensorflow_addons as tfa
# Hyperparameters
batch_size=64
epochs=10
모델 빌드하기
model = tf.keras.Sequential([
tf.keras.layers.Dense(64, input_shape=(784,), activation='relu', name='dense_1'),
tf.keras.layers.Dense(64, activation='relu', name='dense_2'),
tf.keras.layers.Dense(10, activation='softmax', name='predictions'),
])
데이터 준비하기
# Load MNIST dataset as NumPy arrays
dataset = {}
num_validation = 10000
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
# Preprocess the data
x_train = x_train.reshape(-1, 784).astype('float32') / 255
x_test = x_test.reshape(-1, 784).astype('float32') / 255
훈련 및 평가하기
일반적인 keras 옵티마이저를 새로운 tfa 옵티마이저로 간단히 교체합니다.
# Compile the model
model.compile(
optimizer=tfa.optimizers.LazyAdam(0.001), # Utilize TFA optimizer
loss=tf.keras.losses.SparseCategoricalCrossentropy(),
metrics=['accuracy'])
# Train the network
history = model.fit(
x_train,
y_train,
batch_size=batch_size,
epochs=epochs)
Epoch 1/10 1/938 [..............................] - ETA: 0s - loss: 2.3776 - accuracy: 0.0781 28/938 [..............................] - ETA: 1s - loss: 1.7989 - accuracy: 0.4816 56/938 [>.............................] - ETA: 1s - loss: 1.3123 - accuracy: 0.6370 84/938 [=>............................] - ETA: 1s - loss: 1.0495 - accuracy: 0.7078 111/938 [==>...........................] - ETA: 1s - loss: 0.8984 - accuracy: 0.7483 139/938 [===>..........................] - ETA: 1s - loss: 0.7998 - accuracy: 0.7737 166/938 [====>.........................] - ETA: 1s - loss: 0.7200 - accuracy: 0.7965 193/938 [=====>........................] - ETA: 1s - loss: 0.6608 - accuracy: 0.8133 220/938 [======>.......................] - ETA: 1s - loss: 0.6183 - accuracy: 0.8249 248/938 [======>.......................] - ETA: 1s - loss: 0.5896 - accuracy: 0.8327 275/938 [=======>......................] - ETA: 1s - loss: 0.5617 - accuracy: 0.8403 303/938 [========>.....................] - ETA: 1s - loss: 0.5352 - accuracy: 0.8480 331/938 [=========>....................] - ETA: 1s - loss: 0.5133 - accuracy: 0.8545 359/938 [==========>...................] - ETA: 1s - loss: 0.4937 - accuracy: 0.8596 386/938 [===========>..................] - ETA: 1s - loss: 0.4766 - accuracy: 0.8644 414/938 [============>.................] - ETA: 0s - loss: 0.4597 - accuracy: 0.8690 440/938 [=============>................] - ETA: 0s - loss: 0.4462 - accuracy: 0.8728 467/938 [=============>................] - ETA: 0s - loss: 0.4331 - accuracy: 0.8767 494/938 [==============>...............] - ETA: 0s - loss: 0.4246 - accuracy: 0.8788 521/938 [===============>..............] - ETA: 0s - loss: 0.4143 - accuracy: 0.8816 549/938 [================>.............] - ETA: 0s - loss: 0.4043 - accuracy: 0.8844 577/938 [=================>............] - ETA: 0s - loss: 0.3953 - accuracy: 0.8867 604/938 [==================>...........] - ETA: 0s - loss: 0.3869 - accuracy: 0.8891 631/938 [===================>..........] - ETA: 0s - loss: 0.3793 - accuracy: 0.8914 659/938 [====================>.........] - ETA: 0s - loss: 0.3718 - accuracy: 0.8936 686/938 [====================>.........] - ETA: 0s - loss: 0.3650 - accuracy: 0.8953 713/938 [=====================>........] - ETA: 0s - loss: 0.3601 - accuracy: 0.8965 741/938 [======================>.......] - ETA: 0s - loss: 0.3529 - accuracy: 0.8984 767/938 [=======================>......] - ETA: 0s - loss: 0.3475 - accuracy: 0.9001 794/938 [========================>.....] - ETA: 0s - loss: 0.3419 - accuracy: 0.9015 821/938 [=========================>....] - ETA: 0s - loss: 0.3371 - accuracy: 0.9028 849/938 [==========================>...] - ETA: 0s - loss: 0.3311 - accuracy: 0.9046 876/938 [===========================>..] - ETA: 0s - loss: 0.3257 - accuracy: 0.9060 904/938 [===========================>..] - ETA: 0s - loss: 0.3220 - accuracy: 0.9071 932/938 [============================>.] - ETA: 0s - loss: 0.3184 - accuracy: 0.9081 938/938 [==============================] - 2s 2ms/step - loss: 0.3173 - accuracy: 0.9084 Epoch 2/10 1/938 [..............................] - ETA: 0s - loss: 0.0971 - accuracy: 0.9688 29/938 [..............................] - ETA: 1s - loss: 0.1812 - accuracy: 0.9402 56/938 [>.............................] - ETA: 1s - loss: 0.1714 - accuracy: 0.9456 83/938 [=>............................] - ETA: 1s - loss: 0.1694 - accuracy: 0.9482 110/938 [==>...........................] - ETA: 1s - loss: 0.1651 - accuracy: 0.9501 137/938 [===>..........................] - ETA: 1s - loss: 0.1620 - accuracy: 0.9507 165/938 [====>.........................] - ETA: 1s - loss: 0.1623 - accuracy: 0.9506 193/938 [=====>........................] - ETA: 1s - loss: 0.1613 - accuracy: 0.9507 220/938 [======>.......................] - ETA: 1s - loss: 0.1605 - accuracy: 0.9512 248/938 [======>.......................] - ETA: 1s - loss: 0.1578 - accuracy: 0.9517 275/938 [=======>......................] - ETA: 1s - loss: 0.1537 - accuracy: 0.9531 302/938 [========>.....................] - ETA: 1s - loss: 0.1529 - accuracy: 0.9536 330/938 [=========>....................] - ETA: 1s - loss: 0.1530 - accuracy: 0.9538 358/938 [==========>...................] - ETA: 1s - loss: 0.1517 - accuracy: 0.9543 385/938 [===========>..................] - ETA: 1s - loss: 0.1499 - accuracy: 0.9545 412/938 [============>.................] - ETA: 0s - loss: 0.1488 - accuracy: 0.9550 439/938 [=============>................] - ETA: 0s - loss: 0.1492 - accuracy: 0.9553 467/938 [=============>................] - ETA: 0s - loss: 0.1481 - accuracy: 0.9556 495/938 [==============>...............] - ETA: 0s - loss: 0.1495 - accuracy: 0.9553 522/938 [===============>..............] - ETA: 0s - loss: 0.1469 - accuracy: 0.9560 550/938 [================>.............] - ETA: 0s - loss: 0.1475 - accuracy: 0.9559 577/938 [=================>............] - ETA: 0s - loss: 0.1486 - accuracy: 0.9557 604/938 [==================>...........] - ETA: 0s - loss: 0.1469 - accuracy: 0.9562 632/938 [===================>..........] - ETA: 0s - loss: 0.1471 - accuracy: 0.9561 659/938 [====================>.........] - ETA: 0s - loss: 0.1465 - accuracy: 0.9563 687/938 [====================>.........] - ETA: 0s - loss: 0.1448 - accuracy: 0.9568 714/938 [=====================>........] - ETA: 0s - loss: 0.1443 - accuracy: 0.9570 741/938 [======================>.......] - ETA: 0s - loss: 0.1445 - accuracy: 0.9569 769/938 [=======================>......] - ETA: 0s - loss: 0.1438 - accuracy: 0.9569 796/938 [========================>.....] - ETA: 0s - loss: 0.1433 - accuracy: 0.9571 823/938 [=========================>....] - ETA: 0s - loss: 0.1430 - accuracy: 0.9571 851/938 [==========================>...] - ETA: 0s - loss: 0.1421 - accuracy: 0.9573 878/938 [===========================>..] - ETA: 0s - loss: 0.1418 - accuracy: 0.9574 906/938 [===========================>..] - ETA: 0s - loss: 0.1413 - accuracy: 0.9575 934/938 [============================>.] - ETA: 0s - loss: 0.1411 - accuracy: 0.9577 938/938 [==============================] - 2s 2ms/step - loss: 0.1410 - accuracy: 0.9577 Epoch 3/10 1/938 [..............................] - ETA: 0s - loss: 0.0962 - accuracy: 0.9688 28/938 [..............................] - ETA: 1s - loss: 0.1020 - accuracy: 0.9710 56/938 [>.............................] - ETA: 1s - loss: 0.1013 - accuracy: 0.9696 82/938 [=>............................] - ETA: 1s - loss: 0.1050 - accuracy: 0.9697 109/938 [==>...........................] - ETA: 1s - loss: 0.1088 - accuracy: 0.9679 137/938 [===>..........................] - ETA: 1s - loss: 0.1094 - accuracy: 0.9681 165/938 [====>.........................] - ETA: 1s - loss: 0.1070 - accuracy: 0.9687 193/938 [=====>........................] - ETA: 1s - loss: 0.1063 - accuracy: 0.9691 221/938 [======>.......................] - ETA: 1s - loss: 0.1058 - accuracy: 0.9695 250/938 [======>.......................] - ETA: 1s - loss: 0.1067 - accuracy: 0.9696 278/938 [=======>......................] - ETA: 1s - loss: 0.1062 - accuracy: 0.9695 306/938 [========>.....................] - ETA: 1s - loss: 0.1039 - accuracy: 0.9701 334/938 [=========>....................] - ETA: 1s - loss: 0.1037 - accuracy: 0.9703 361/938 [==========>...................] - ETA: 1s - loss: 0.1040 - accuracy: 0.9702 389/938 [===========>..................] - ETA: 1s - loss: 0.1037 - accuracy: 0.9700 417/938 [============>.................] - ETA: 0s - loss: 0.1023 - accuracy: 0.9702 444/938 [=============>................] - ETA: 0s - loss: 0.1020 - accuracy: 0.9703 472/938 [==============>...............] - ETA: 0s - loss: 0.1018 - accuracy: 0.9700 499/938 [==============>...............] - ETA: 0s - loss: 0.1027 - accuracy: 0.9696 527/938 [===============>..............] - ETA: 0s - loss: 0.1024 - accuracy: 0.9698 555/938 [================>.............] - ETA: 0s - loss: 0.1025 - accuracy: 0.9700 583/938 [=================>............] - ETA: 0s - loss: 0.1024 - accuracy: 0.9700 610/938 [==================>...........] - ETA: 0s - loss: 0.1024 - accuracy: 0.9700 637/938 [===================>..........] - ETA: 0s - loss: 0.1029 - accuracy: 0.9698 664/938 [====================>.........] - ETA: 0s - loss: 0.1024 - accuracy: 0.9697 691/938 [=====================>........] - ETA: 0s - loss: 0.1018 - accuracy: 0.9700 719/938 [=====================>........] - ETA: 0s - loss: 0.1016 - accuracy: 0.9700 748/938 [======================>.......] - ETA: 0s - loss: 0.1019 - accuracy: 0.9697 777/938 [=======================>......] - ETA: 0s - loss: 0.1015 - accuracy: 0.9698 805/938 [========================>.....] - ETA: 0s - loss: 0.1014 - accuracy: 0.9698 833/938 [=========================>....] - ETA: 0s - loss: 0.1013 - accuracy: 0.9696 861/938 [==========================>...] - ETA: 0s - loss: 0.1016 - accuracy: 0.9694 888/938 [===========================>..] - ETA: 0s - loss: 0.1015 - accuracy: 0.9695 916/938 [============================>.] - ETA: 0s - loss: 0.1008 - accuracy: 0.9697 938/938 [==============================] - 2s 2ms/step - loss: 0.1009 - accuracy: 0.9696 Epoch 4/10 1/938 [..............................] - ETA: 0s - loss: 0.0588 - accuracy: 0.9688 29/938 [..............................] - ETA: 1s - loss: 0.0897 - accuracy: 0.9736 56/938 [>.............................] - ETA: 1s - loss: 0.0921 - accuracy: 0.9718 83/938 [=>............................] - ETA: 1s - loss: 0.0900 - accuracy: 0.9727 110/938 [==>...........................] - ETA: 1s - loss: 0.0837 - accuracy: 0.9751 137/938 [===>..........................] - ETA: 1s - loss: 0.0815 - accuracy: 0.9749 165/938 [====>.........................] - ETA: 1s - loss: 0.0827 - accuracy: 0.9744 193/938 [=====>........................] - ETA: 1s - loss: 0.0837 - accuracy: 0.9738 220/938 [======>.......................] - ETA: 1s - loss: 0.0830 - accuracy: 0.9738 248/938 [======>.......................] - ETA: 1s - loss: 0.0839 - accuracy: 0.9737 275/938 [=======>......................] - ETA: 1s - loss: 0.0825 - accuracy: 0.9744 300/938 [========>.....................] - ETA: 1s - loss: 0.0807 - accuracy: 0.9752 326/938 [=========>....................] - ETA: 1s - loss: 0.0809 - accuracy: 0.9752 352/938 [==========>...................] - ETA: 1s - loss: 0.0808 - accuracy: 0.9750 379/938 [===========>..................] - ETA: 1s - loss: 0.0817 - accuracy: 0.9748 407/938 [============>.................] - ETA: 0s - loss: 0.0813 - accuracy: 0.9751 435/938 [============>.................] - ETA: 0s - loss: 0.0828 - accuracy: 0.9745 462/938 [=============>................] - ETA: 0s - loss: 0.0820 - accuracy: 0.9747 490/938 [==============>...............] - ETA: 0s - loss: 0.0822 - accuracy: 0.9746 518/938 [===============>..............] - ETA: 0s - loss: 0.0815 - accuracy: 0.9748 544/938 [================>.............] - ETA: 0s - loss: 0.0808 - accuracy: 0.9750 570/938 [=================>............] - ETA: 0s - loss: 0.0805 - accuracy: 0.9750 598/938 [==================>...........] - ETA: 0s - loss: 0.0802 - accuracy: 0.9752 625/938 [==================>...........] - ETA: 0s - loss: 0.0800 - accuracy: 0.9754 652/938 [===================>..........] - ETA: 0s - loss: 0.0802 - accuracy: 0.9754 680/938 [====================>.........] - ETA: 0s - loss: 0.0808 - accuracy: 0.9753 708/938 [=====================>........] - ETA: 0s - loss: 0.0812 - accuracy: 0.9751 735/938 [======================>.......] - ETA: 0s - loss: 0.0814 - accuracy: 0.9750 763/938 [=======================>......] - ETA: 0s - loss: 0.0808 - accuracy: 0.9752 791/938 [========================>.....] - ETA: 0s - loss: 0.0805 - accuracy: 0.9752 818/938 [=========================>....] - ETA: 0s - loss: 0.0801 - accuracy: 0.9753 845/938 [==========================>...] - ETA: 0s - loss: 0.0807 - accuracy: 0.9752 873/938 [==========================>...] - ETA: 0s - loss: 0.0810 - accuracy: 0.9752 901/938 [===========================>..] - ETA: 0s - loss: 0.0809 - accuracy: 0.9750 929/938 [============================>.] - ETA: 0s - loss: 0.0812 - accuracy: 0.9751 938/938 [==============================] - 2s 2ms/step - loss: 0.0814 - accuracy: 0.9750 Epoch 5/10 1/938 [..............................] - ETA: 0s - loss: 0.1086 - accuracy: 0.9688 29/938 [..............................] - ETA: 1s - loss: 0.0731 - accuracy: 0.9784 57/938 [>.............................] - ETA: 1s - loss: 0.0684 - accuracy: 0.9805 85/938 [=>............................] - ETA: 1s - loss: 0.0699 - accuracy: 0.9790 113/938 [==>...........................] - ETA: 1s - loss: 0.0668 - accuracy: 0.9794 140/938 [===>..........................] - ETA: 1s - loss: 0.0652 - accuracy: 0.9795 168/938 [====>.........................] - ETA: 1s - loss: 0.0652 - accuracy: 0.9795 195/938 [=====>........................] - ETA: 1s - loss: 0.0645 - accuracy: 0.9796 222/938 [======>.......................] - ETA: 1s - loss: 0.0646 - accuracy: 0.9797 249/938 [======>.......................] - ETA: 1s - loss: 0.0638 - accuracy: 0.9800 276/938 [=======>......................] - ETA: 1s - loss: 0.0624 - accuracy: 0.9805 304/938 [========>.....................] - ETA: 1s - loss: 0.0626 - accuracy: 0.9805 332/938 [=========>....................] - ETA: 1s - loss: 0.0617 - accuracy: 0.9809 360/938 [==========>...................] - ETA: 1s - loss: 0.0626 - accuracy: 0.9808 388/938 [===========>..................] - ETA: 1s - loss: 0.0621 - accuracy: 0.9809 416/938 [============>.................] - ETA: 0s - loss: 0.0629 - accuracy: 0.9807 444/938 [=============>................] - ETA: 0s - loss: 0.0627 - accuracy: 0.9805 471/938 [==============>...............] - ETA: 0s - loss: 0.0630 - accuracy: 0.9805 498/938 [==============>...............] - ETA: 0s - loss: 0.0637 - accuracy: 0.9801 525/938 [===============>..............] - ETA: 0s - loss: 0.0642 - accuracy: 0.9799 552/938 [================>.............] - ETA: 0s - loss: 0.0654 - accuracy: 0.9794 580/938 [=================>............] - ETA: 0s - loss: 0.0659 - accuracy: 0.9793 608/938 [==================>...........] - ETA: 0s - loss: 0.0654 - accuracy: 0.9794 637/938 [===================>..........] - ETA: 0s - loss: 0.0652 - accuracy: 0.9794 664/938 [====================>.........] - ETA: 0s - loss: 0.0656 - accuracy: 0.9792 692/938 [=====================>........] - ETA: 0s - loss: 0.0655 - accuracy: 0.9792 719/938 [=====================>........] - ETA: 0s - loss: 0.0653 - accuracy: 0.9794 747/938 [======================>.......] - ETA: 0s - loss: 0.0653 - accuracy: 0.9795 774/938 [=======================>......] - ETA: 0s - loss: 0.0663 - accuracy: 0.9790 802/938 [========================>.....] - ETA: 0s - loss: 0.0665 - accuracy: 0.9791 830/938 [=========================>....] - ETA: 0s - loss: 0.0663 - accuracy: 0.9791 858/938 [==========================>...] - ETA: 0s - loss: 0.0662 - accuracy: 0.9791 886/938 [===========================>..] - ETA: 0s - loss: 0.0663 - accuracy: 0.9791 912/938 [============================>.] - ETA: 0s - loss: 0.0663 - accuracy: 0.9791 938/938 [==============================] - 2s 2ms/step - loss: 0.0663 - accuracy: 0.9790 Epoch 6/10 1/938 [..............................] - ETA: 0s - loss: 0.0273 - accuracy: 1.0000 29/938 [..............................] - ETA: 1s - loss: 0.0567 - accuracy: 0.9849 56/938 [>.............................] - ETA: 1s - loss: 0.0520 - accuracy: 0.9852 83/938 [=>............................] - ETA: 1s - loss: 0.0516 - accuracy: 0.9855 111/938 [==>...........................] - ETA: 1s - loss: 0.0531 - accuracy: 0.9848 137/938 [===>..........................] - ETA: 1s - loss: 0.0553 - accuracy: 0.9838 164/938 [====>.........................] - ETA: 1s - loss: 0.0539 - accuracy: 0.9837 191/938 [=====>........................] - ETA: 1s - loss: 0.0536 - accuracy: 0.9836 218/938 [=====>........................] - ETA: 1s - loss: 0.0530 - accuracy: 0.9842 246/938 [======>.......................] - ETA: 1s - loss: 0.0530 - accuracy: 0.9841 273/938 [=======>......................] - ETA: 1s - loss: 0.0534 - accuracy: 0.9841 300/938 [========>.....................] - ETA: 1s - loss: 0.0525 - accuracy: 0.9842 327/938 [=========>....................] - ETA: 1s - loss: 0.0519 - accuracy: 0.9841 355/938 [==========>...................] - ETA: 1s - loss: 0.0526 - accuracy: 0.9840 380/938 [===========>..................] - ETA: 1s - loss: 0.0530 - accuracy: 0.9838 407/938 [============>.................] - ETA: 1s - loss: 0.0529 - accuracy: 0.9836 434/938 [============>.................] - ETA: 0s - loss: 0.0523 - accuracy: 0.9838 461/938 [=============>................] - ETA: 0s - loss: 0.0525 - accuracy: 0.9837 489/938 [==============>...............] - ETA: 0s - loss: 0.0538 - accuracy: 0.9833 517/938 [===============>..............] - ETA: 0s - loss: 0.0534 - accuracy: 0.9833 544/938 [================>.............] - ETA: 0s - loss: 0.0533 - accuracy: 0.9833 571/938 [=================>............] - ETA: 0s - loss: 0.0540 - accuracy: 0.9832 598/938 [==================>...........] - ETA: 0s - loss: 0.0535 - accuracy: 0.9833 626/938 [===================>..........] - ETA: 0s - loss: 0.0537 - accuracy: 0.9832 654/938 [===================>..........] - ETA: 0s - loss: 0.0540 - accuracy: 0.9831 682/938 [====================>.........] - ETA: 0s - loss: 0.0540 - accuracy: 0.9831 710/938 [=====================>........] - ETA: 0s - loss: 0.0539 - accuracy: 0.9831 738/938 [======================>.......] - ETA: 0s - loss: 0.0542 - accuracy: 0.9830 766/938 [=======================>......] - ETA: 0s - loss: 0.0542 - accuracy: 0.9829 794/938 [========================>.....] - ETA: 0s - loss: 0.0538 - accuracy: 0.9830 821/938 [=========================>....] - ETA: 0s - loss: 0.0537 - accuracy: 0.9830 848/938 [==========================>...] - ETA: 0s - loss: 0.0536 - accuracy: 0.9829 876/938 [===========================>..] - ETA: 0s - loss: 0.0542 - accuracy: 0.9827 904/938 [===========================>..] - ETA: 0s - loss: 0.0541 - accuracy: 0.9827 931/938 [============================>.] - ETA: 0s - loss: 0.0539 - accuracy: 0.9827 938/938 [==============================] - 2s 2ms/step - loss: 0.0539 - accuracy: 0.9828 Epoch 7/10 1/938 [..............................] - ETA: 0s - loss: 0.1814 - accuracy: 0.9375 28/938 [..............................] - ETA: 1s - loss: 0.0421 - accuracy: 0.9877 55/938 [>.............................] - ETA: 1s - loss: 0.0441 - accuracy: 0.9869 82/938 [=>............................] - ETA: 1s - loss: 0.0454 - accuracy: 0.9863 110/938 [==>...........................] - ETA: 1s - loss: 0.0456 - accuracy: 0.9861 137/938 [===>..........................] - ETA: 1s - loss: 0.0477 - accuracy: 0.9863 163/938 [====>.........................] - ETA: 1s - loss: 0.0478 - accuracy: 0.9859 190/938 [=====>........................] - ETA: 1s - loss: 0.0475 - accuracy: 0.9861 218/938 [=====>........................] - ETA: 1s - loss: 0.0471 - accuracy: 0.9862 245/938 [======>.......................] - ETA: 1s - loss: 0.0478 - accuracy: 0.9859 272/938 [=======>......................] - ETA: 1s - loss: 0.0478 - accuracy: 0.9859 299/938 [========>.....................] - ETA: 1s - loss: 0.0472 - accuracy: 0.9860 326/938 [=========>....................] - ETA: 1s - loss: 0.0475 - accuracy: 0.9860 352/938 [==========>...................] - ETA: 1s - loss: 0.0479 - accuracy: 0.9858 378/938 [===========>..................] - ETA: 1s - loss: 0.0470 - accuracy: 0.9861 405/938 [===========>..................] - ETA: 1s - loss: 0.0462 - accuracy: 0.9860 431/938 [============>.................] - ETA: 0s - loss: 0.0461 - accuracy: 0.9859 456/938 [=============>................] - ETA: 0s - loss: 0.0458 - accuracy: 0.9859 483/938 [==============>...............] - ETA: 0s - loss: 0.0462 - accuracy: 0.9856 509/938 [===============>..............] - ETA: 0s - loss: 0.0460 - accuracy: 0.9856 534/938 [================>.............] - ETA: 0s - loss: 0.0459 - accuracy: 0.9856 560/938 [================>.............] - ETA: 0s - loss: 0.0455 - accuracy: 0.9854 587/938 [=================>............] - ETA: 0s - loss: 0.0453 - accuracy: 0.9855 613/938 [==================>...........] - ETA: 0s - loss: 0.0456 - accuracy: 0.9854 640/938 [===================>..........] - ETA: 0s - loss: 0.0458 - accuracy: 0.9853 667/938 [====================>.........] - ETA: 0s - loss: 0.0465 - accuracy: 0.9851 693/938 [=====================>........] - ETA: 0s - loss: 0.0465 - accuracy: 0.9850 720/938 [======================>.......] - ETA: 0s - loss: 0.0466 - accuracy: 0.9848 747/938 [======================>.......] - ETA: 0s - loss: 0.0465 - accuracy: 0.9849 773/938 [=======================>......] - ETA: 0s - loss: 0.0465 - accuracy: 0.9849 799/938 [========================>.....] - ETA: 0s - loss: 0.0467 - accuracy: 0.9849 827/938 [=========================>....] - ETA: 0s - loss: 0.0464 - accuracy: 0.9850 855/938 [==========================>...] - ETA: 0s - loss: 0.0467 - accuracy: 0.9850 882/938 [===========================>..] - ETA: 0s - loss: 0.0467 - accuracy: 0.9851 908/938 [============================>.] - ETA: 0s - loss: 0.0467 - accuracy: 0.9851 934/938 [============================>.] - ETA: 0s - loss: 0.0468 - accuracy: 0.9850 938/938 [==============================] - 2s 2ms/step - loss: 0.0467 - accuracy: 0.9851 Epoch 8/10 1/938 [..............................] - ETA: 0s - loss: 0.0864 - accuracy: 0.9844 27/938 [..............................] - ETA: 1s - loss: 0.0365 - accuracy: 0.9890 55/938 [>.............................] - ETA: 1s - loss: 0.0304 - accuracy: 0.9909 81/938 [=>............................] - ETA: 1s - loss: 0.0316 - accuracy: 0.9902 107/938 [==>...........................] - ETA: 1s - loss: 0.0351 - accuracy: 0.9892 133/938 [===>..........................] - ETA: 1s - loss: 0.0351 - accuracy: 0.9893 159/938 [====>.........................] - ETA: 1s - loss: 0.0358 - accuracy: 0.9887 185/938 [====>.........................] - ETA: 1s - loss: 0.0349 - accuracy: 0.9892 211/938 [=====>........................] - ETA: 1s - loss: 0.0350 - accuracy: 0.9894 237/938 [======>.......................] - ETA: 1s - loss: 0.0340 - accuracy: 0.9898 263/938 [=======>......................] - ETA: 1s - loss: 0.0358 - accuracy: 0.9892 289/938 [========>.....................] - ETA: 1s - loss: 0.0348 - accuracy: 0.9897 315/938 [=========>....................] - ETA: 1s - loss: 0.0348 - accuracy: 0.9896 341/938 [=========>....................] - ETA: 1s - loss: 0.0352 - accuracy: 0.9895 367/938 [==========>...................] - ETA: 1s - loss: 0.0353 - accuracy: 0.9894 393/938 [===========>..................] - ETA: 1s - loss: 0.0364 - accuracy: 0.9889 420/938 [============>.................] - ETA: 1s - loss: 0.0361 - accuracy: 0.9889 446/938 [=============>................] - ETA: 0s - loss: 0.0354 - accuracy: 0.9890 473/938 [==============>...............] - ETA: 0s - loss: 0.0356 - accuracy: 0.9889 499/938 [==============>...............] - ETA: 0s - loss: 0.0358 - accuracy: 0.9887 524/938 [===============>..............] - ETA: 0s - loss: 0.0368 - accuracy: 0.9885 549/938 [================>.............] - ETA: 0s - loss: 0.0370 - accuracy: 0.9885 575/938 [=================>............] - ETA: 0s - loss: 0.0371 - accuracy: 0.9884 602/938 [==================>...........] - ETA: 0s - loss: 0.0381 - accuracy: 0.9882 628/938 [===================>..........] - ETA: 0s - loss: 0.0381 - accuracy: 0.9882 654/938 [===================>..........] - ETA: 0s - loss: 0.0384 - accuracy: 0.9881 679/938 [====================>.........] - ETA: 0s - loss: 0.0391 - accuracy: 0.9879 705/938 [=====================>........] - ETA: 0s - loss: 0.0390 - accuracy: 0.9879 731/938 [======================>.......] - ETA: 0s - loss: 0.0398 - accuracy: 0.9876 757/938 [=======================>......] - ETA: 0s - loss: 0.0401 - accuracy: 0.9874 784/938 [========================>.....] - ETA: 0s - loss: 0.0401 - accuracy: 0.9874 810/938 [========================>.....] - ETA: 0s - loss: 0.0402 - accuracy: 0.9874 836/938 [=========================>....] - ETA: 0s - loss: 0.0402 - accuracy: 0.9874 863/938 [==========================>...] - ETA: 0s - loss: 0.0402 - accuracy: 0.9873 889/938 [===========================>..] - ETA: 0s - loss: 0.0403 - accuracy: 0.9873 914/938 [============================>.] - ETA: 0s - loss: 0.0403 - accuracy: 0.9873 938/938 [==============================] - 2s 2ms/step - loss: 0.0402 - accuracy: 0.9873 Epoch 9/10 1/938 [..............................] - ETA: 0s - loss: 0.0204 - accuracy: 1.0000 28/938 [..............................] - ETA: 1s - loss: 0.0271 - accuracy: 0.9883 55/938 [>.............................] - ETA: 1s - loss: 0.0254 - accuracy: 0.9898 81/938 [=>............................] - ETA: 1s - loss: 0.0251 - accuracy: 0.9902 107/938 [==>...........................] - ETA: 1s - loss: 0.0268 - accuracy: 0.9899 133/938 [===>..........................] - ETA: 1s - loss: 0.0285 - accuracy: 0.9893 159/938 [====>.........................] - ETA: 1s - loss: 0.0291 - accuracy: 0.9895 185/938 [====>.........................] - ETA: 1s - loss: 0.0282 - accuracy: 0.9899 213/938 [=====>........................] - ETA: 1s - loss: 0.0275 - accuracy: 0.9905 240/938 [======>.......................] - ETA: 1s - loss: 0.0274 - accuracy: 0.9907 267/938 [=======>......................] - ETA: 1s - loss: 0.0270 - accuracy: 0.9908 294/938 [========>.....................] - ETA: 1s - loss: 0.0272 - accuracy: 0.9907 321/938 [=========>....................] - ETA: 1s - loss: 0.0281 - accuracy: 0.9902 347/938 [==========>...................] - ETA: 1s - loss: 0.0279 - accuracy: 0.9905 375/938 [==========>...................] - ETA: 1s - loss: 0.0287 - accuracy: 0.9902 403/938 [===========>..................] - ETA: 1s - loss: 0.0296 - accuracy: 0.9901 431/938 [============>.................] - ETA: 0s - loss: 0.0305 - accuracy: 0.9897 458/938 [=============>................] - ETA: 0s - loss: 0.0305 - accuracy: 0.9897 486/938 [==============>...............] - ETA: 0s - loss: 0.0313 - accuracy: 0.9894 513/938 [===============>..............] - ETA: 0s - loss: 0.0317 - accuracy: 0.9892 540/938 [================>.............] - ETA: 0s - loss: 0.0317 - accuracy: 0.9891 568/938 [=================>............] - ETA: 0s - loss: 0.0317 - accuracy: 0.9891 596/938 [==================>...........] - ETA: 0s - loss: 0.0324 - accuracy: 0.9889 624/938 [==================>...........] - ETA: 0s - loss: 0.0329 - accuracy: 0.9888 652/938 [===================>..........] - ETA: 0s - loss: 0.0329 - accuracy: 0.9889 680/938 [====================>.........] - ETA: 0s - loss: 0.0329 - accuracy: 0.9889 707/938 [=====================>........] - ETA: 0s - loss: 0.0329 - accuracy: 0.9889 734/938 [======================>.......] - ETA: 0s - loss: 0.0333 - accuracy: 0.9890 761/938 [=======================>......] - ETA: 0s - loss: 0.0329 - accuracy: 0.9892 787/938 [========================>.....] - ETA: 0s - loss: 0.0328 - accuracy: 0.9891 814/938 [=========================>....] - ETA: 0s - loss: 0.0332 - accuracy: 0.9890 841/938 [=========================>....] - ETA: 0s - loss: 0.0335 - accuracy: 0.9888 869/938 [==========================>...] - ETA: 0s - loss: 0.0336 - accuracy: 0.9888 897/938 [===========================>..] - ETA: 0s - loss: 0.0339 - accuracy: 0.9887 925/938 [============================>.] - ETA: 0s - loss: 0.0346 - accuracy: 0.9884 938/938 [==============================] - 2s 2ms/step - loss: 0.0348 - accuracy: 0.9883 Epoch 10/10 1/938 [..............................] - ETA: 0s - loss: 0.0145 - accuracy: 1.0000 29/938 [..............................] - ETA: 1s - loss: 0.0265 - accuracy: 0.9914 57/938 [>.............................] - ETA: 1s - loss: 0.0235 - accuracy: 0.9923 85/938 [=>............................] - ETA: 1s - loss: 0.0242 - accuracy: 0.9923 113/938 [==>...........................] - ETA: 1s - loss: 0.0264 - accuracy: 0.9914 141/938 [===>..........................] - ETA: 1s - loss: 0.0296 - accuracy: 0.9904 168/938 [====>.........................] - ETA: 1s - loss: 0.0301 - accuracy: 0.9903 195/938 [=====>........................] - ETA: 1s - loss: 0.0297 - accuracy: 0.9906 223/938 [======>.......................] - ETA: 1s - loss: 0.0304 - accuracy: 0.9903 250/938 [======>.......................] - ETA: 1s - loss: 0.0293 - accuracy: 0.9906 277/938 [=======>......................] - ETA: 1s - loss: 0.0300 - accuracy: 0.9904 304/938 [========>.....................] - ETA: 1s - loss: 0.0291 - accuracy: 0.9905 332/938 [=========>....................] - ETA: 1s - loss: 0.0281 - accuracy: 0.9909 359/938 [==========>...................] - ETA: 1s - loss: 0.0281 - accuracy: 0.9907 386/938 [===========>..................] - ETA: 1s - loss: 0.0288 - accuracy: 0.9904 413/938 [============>.................] - ETA: 0s - loss: 0.0285 - accuracy: 0.9904 440/938 [=============>................] - ETA: 0s - loss: 0.0287 - accuracy: 0.9904 468/938 [=============>................] - ETA: 0s - loss: 0.0281 - accuracy: 0.9906 495/938 [==============>...............] - ETA: 0s - loss: 0.0277 - accuracy: 0.9907 522/938 [===============>..............] - ETA: 0s - loss: 0.0279 - accuracy: 0.9907 550/938 [================>.............] - ETA: 0s - loss: 0.0277 - accuracy: 0.9907 578/938 [=================>............] - ETA: 0s - loss: 0.0276 - accuracy: 0.9908 606/938 [==================>...........] - ETA: 0s - loss: 0.0280 - accuracy: 0.9907 634/938 [===================>..........] - ETA: 0s - loss: 0.0279 - accuracy: 0.9908 662/938 [====================>.........] - ETA: 0s - loss: 0.0289 - accuracy: 0.9905 690/938 [=====================>........] - ETA: 0s - loss: 0.0294 - accuracy: 0.9903 718/938 [=====================>........] - ETA: 0s - loss: 0.0301 - accuracy: 0.9901 745/938 [======================>.......] - ETA: 0s - loss: 0.0301 - accuracy: 0.9900 772/938 [=======================>......] - ETA: 0s - loss: 0.0300 - accuracy: 0.9900 800/938 [========================>.....] - ETA: 0s - loss: 0.0302 - accuracy: 0.9900 827/938 [=========================>....] - ETA: 0s - loss: 0.0302 - accuracy: 0.9901 855/938 [==========================>...] - ETA: 0s - loss: 0.0301 - accuracy: 0.9901 882/938 [===========================>..] - ETA: 0s - loss: 0.0297 - accuracy: 0.9902 909/938 [============================>.] - ETA: 0s - loss: 0.0295 - accuracy: 0.9903 937/938 [============================>.] - ETA: 0s - loss: 0.0297 - accuracy: 0.9903 938/938 [==============================] - 2s 2ms/step - loss: 0.0297 - accuracy: 0.9903
# Evaluate the network
print('Evaluate on test data:')
results = model.evaluate(x_test, y_test, batch_size=128, verbose = 2)
print('Test loss = {0}, Test acc: {1}'.format(results[0], results[1]))
Evaluate on test data: 79/79 - 0s - loss: 0.0851 - accuracy: 0.9754 Test loss = 0.08514752238988876, Test acc: 0.9753999710083008