 在 TensorFlow.org 上查看 |
 在 Google Colab 中运行 |
 在 GitHub 上查看源代码 |
 下载笔记本 |
概述
此笔记本将演示如何使用 TensorFlow Addons 中的 TimeStopping 回调。
设置
pip install -U tensorflow-addonsimport tensorflow_addons as tfa
from tensorflow.keras.datasets import mnist
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Flatten
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow_addons/utils/ensure_tf_install.py:67: UserWarning: Tensorflow Addons supports using Python ops for all Tensorflow versions above or equal to 2.3.0 and strictly below 2.6.0 (nightly versions are not supported). The versions of TensorFlow you are currently using is 2.6.0 and is not supported. Some things might work, some things might not. If you were to encounter a bug, do not file an issue. If you want to make sure you're using a tested and supported configuration, either change the TensorFlow version or the TensorFlow Addons's version. You can find the compatibility matrix in TensorFlow Addon's readme: https://github.com/tensorflow/addons UserWarning,
导入并归一化数据
# the data, split between train and test sets
(x_train, y_train), (x_test, y_test) = mnist.load_data()
# normalize data
x_train, x_test = x_train / 255.0, x_test / 255.0
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz 16384/11490434 [..............................] - ETA: 0s 9527296/11490434 [=======================>......] - ETA: 0s 11493376/11490434 [==============================] - 0s 0us/step 11501568/11490434 [==============================] - 0s 0us/step
构建简单的 MNIST CNN 模型
# build the model using the Sequential API
model = Sequential()
model.add(Flatten(input_shape=(28, 28)))
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(10, activation='softmax'))
model.compile(optimizer='adam',
loss = 'sparse_categorical_crossentropy',
metrics=['accuracy'])
2021-08-13 19:51:38.498285: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-08-13 19:51:38.506150: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-08-13 19:51:38.507036: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-08-13 19:51:38.508667: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2021-08-13 19:51:38.509232: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-08-13 19:51:38.510194: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-08-13 19:51:38.511098: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-08-13 19:51:39.093394: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-08-13 19:51:39.094491: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-08-13 19:51:39.095386: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2021-08-13 19:51:39.096269: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory: -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0
简单的 TimeStopping 用法
# initialize TimeStopping callback
time_stopping_callback = tfa.callbacks.TimeStopping(seconds=5, verbose=1)
# train the model with tqdm_callback
# make sure to set verbose = 0 to disable
# the default progress bar.
model.fit(x_train, y_train,
batch_size=64,
epochs=100,
callbacks=[time_stopping_callback],
validation_data=(x_test, y_test))
2021-08-13 19:51:40.120947: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2) Epoch 1/100 1/938 [..............................] - ETA: 12:24 - loss: 2.4173 - accuracy: 0.0781 26/938 [..............................] - ETA: 1s - loss: 1.5276 - accuracy: 0.5793 52/938 [>.............................] - ETA: 1s - loss: 1.1426 - accuracy: 0.6830 79/938 [=>............................] - ETA: 1s - loss: 0.9382 - accuracy: 0.7385 106/938 [==>...........................] - ETA: 1s - loss: 0.8153 - accuracy: 0.7721 133/938 [===>..........................] - ETA: 1s - loss: 0.7410 - accuracy: 0.7924 160/938 [====>.........................] - ETA: 1s - loss: 0.6846 - accuracy: 0.8074 186/938 [====>.........................] - ETA: 1s - loss: 0.6423 - accuracy: 0.8190 212/938 [=====>........................] - ETA: 1s - loss: 0.6087 - accuracy: 0.8278 239/938 [======>.......................] - ETA: 1s - loss: 0.5817 - accuracy: 0.8347 265/938 [=======>......................] - ETA: 1s - loss: 0.5579 - accuracy: 0.8410 292/938 [========>.....................] - ETA: 1s - loss: 0.5343 - accuracy: 0.8470 319/938 [=========>....................] - ETA: 1s - loss: 0.5145 - accuracy: 0.8527 345/938 [==========>...................] - ETA: 1s - loss: 0.4998 - accuracy: 0.8568 372/938 [==========>...................] - ETA: 1s - loss: 0.4861 - accuracy: 0.8611 400/938 [===========>..................] - ETA: 1s - loss: 0.4719 - accuracy: 0.8654 426/938 [============>.................] - ETA: 0s - loss: 0.4602 - accuracy: 0.8688 453/938 [=============>................] - ETA: 0s - loss: 0.4486 - accuracy: 0.8720 479/938 [==============>...............] - ETA: 0s - loss: 0.4386 - accuracy: 0.8747 506/938 [===============>..............] - ETA: 0s - loss: 0.4301 - accuracy: 0.8771 531/938 [===============>..............] - ETA: 0s - loss: 0.4219 - accuracy: 0.8797 558/938 [================>.............] - ETA: 0s - loss: 0.4143 - accuracy: 0.8819 585/938 [=================>............] - ETA: 0s - loss: 0.4051 - accuracy: 0.8842 611/938 [==================>...........] - ETA: 0s - loss: 0.3983 - accuracy: 0.8861 638/938 [===================>..........] - ETA: 0s - loss: 0.3923 - accuracy: 0.8878 665/938 [====================>.........] - ETA: 0s - loss: 0.3855 - accuracy: 0.8898 691/938 [=====================>........] - ETA: 0s - loss: 0.3804 - accuracy: 0.8910 718/938 [=====================>........] - ETA: 0s - loss: 0.3746 - accuracy: 0.8927 745/938 [======================>.......] - ETA: 0s - loss: 0.3696 - accuracy: 0.8945 772/938 [=======================>......] - ETA: 0s - loss: 0.3643 - accuracy: 0.8959 799/938 [========================>.....] - ETA: 0s - loss: 0.3600 - accuracy: 0.8969 825/938 [=========================>....] - ETA: 0s - loss: 0.3546 - accuracy: 0.8986 852/938 [==========================>...] - ETA: 0s - loss: 0.3505 - accuracy: 0.8997 880/938 [===========================>..] - ETA: 0s - loss: 0.3463 - accuracy: 0.9009 907/938 [============================>.] - ETA: 0s - loss: 0.3422 - accuracy: 0.9019 934/938 [============================>.] - ETA: 0s - loss: 0.3380 - accuracy: 0.9031 938/938 [==============================] - 3s 2ms/step - loss: 0.3375 - accuracy: 0.9032 - val_loss: 0.1676 - val_accuracy: 0.9500 Epoch 2/100 1/938 [..............................] - ETA: 2s - loss: 0.2030 - accuracy: 0.9219 27/938 [..............................] - ETA: 1s - loss: 0.1782 - accuracy: 0.9502 53/938 [>.............................] - ETA: 1s - loss: 0.1839 - accuracy: 0.9496 80/938 [=>............................] - ETA: 1s - loss: 0.1778 - accuracy: 0.9496 108/938 [==>...........................] - ETA: 1s - loss: 0.1750 - accuracy: 0.9512 135/938 [===>..........................] - ETA: 1s - loss: 0.1706 - accuracy: 0.9523 161/938 [====>.........................] - ETA: 1s - loss: 0.1715 - accuracy: 0.9524 187/938 [====>.........................] - ETA: 1s - loss: 0.1765 - accuracy: 0.9505 214/938 [=====>........................] - ETA: 1s - loss: 0.1729 - accuracy: 0.9512 241/938 [======>.......................] - ETA: 1s - loss: 0.1736 - accuracy: 0.9505 267/938 [=======>......................] - ETA: 1s - loss: 0.1759 - accuracy: 0.9502 294/938 [========>.....................] - ETA: 1s - loss: 0.1750 - accuracy: 0.9505 320/938 [=========>....................] - ETA: 1s - loss: 0.1747 - accuracy: 0.9508 348/938 [==========>...................] - ETA: 1s - loss: 0.1755 - accuracy: 0.9503 375/938 [==========>...................] - ETA: 1s - loss: 0.1755 - accuracy: 0.9503 402/938 [===========>..................] - ETA: 1s - loss: 0.1754 - accuracy: 0.9506 428/938 [============>.................] - ETA: 0s - loss: 0.1733 - accuracy: 0.9512 455/938 [=============>................] - ETA: 0s - loss: 0.1729 - accuracy: 0.9513 482/938 [==============>...............] - ETA: 0s - loss: 0.1716 - accuracy: 0.9517 509/938 [===============>..............] - ETA: 0s - loss: 0.1724 - accuracy: 0.9511 535/938 [================>.............] - ETA: 0s - loss: 0.1707 - accuracy: 0.9513 561/938 [================>.............] - ETA: 0s - loss: 0.1690 - accuracy: 0.9518 588/938 [=================>............] - ETA: 0s - loss: 0.1693 - accuracy: 0.9514 615/938 [==================>...........] - ETA: 0s - loss: 0.1690 - accuracy: 0.9513 642/938 [===================>..........] - ETA: 0s - loss: 0.1676 - accuracy: 0.9517 670/938 [====================>.........] - ETA: 0s - loss: 0.1671 - accuracy: 0.9518 696/938 [=====================>........] - ETA: 0s - loss: 0.1662 - accuracy: 0.9519 724/938 [======================>.......] - ETA: 0s - loss: 0.1652 - accuracy: 0.9520 752/938 [=======================>......] - ETA: 0s - loss: 0.1643 - accuracy: 0.9524 779/938 [=======================>......] - ETA: 0s - loss: 0.1641 - accuracy: 0.9524 805/938 [========================>.....] - ETA: 0s - loss: 0.1637 - accuracy: 0.9524 832/938 [=========================>....] - ETA: 0s - loss: 0.1636 - accuracy: 0.9523 859/938 [==========================>...] - ETA: 0s - loss: 0.1629 - accuracy: 0.9526 885/938 [===========================>..] - ETA: 0s - loss: 0.1624 - accuracy: 0.9526 912/938 [============================>.] - ETA: 0s - loss: 0.1615 - accuracy: 0.9530 938/938 [==============================] - 2s 2ms/step - loss: 0.1608 - accuracy: 0.9532 - val_loss: 0.1146 - val_accuracy: 0.9639 Epoch 3/100 1/938 [..............................] - ETA: 2s - loss: 0.2198 - accuracy: 0.9062 28/938 [..............................] - ETA: 1s - loss: 0.1323 - accuracy: 0.9581 55/938 [>.............................] - ETA: 1s - loss: 0.1227 - accuracy: 0.9614 83/938 [=>............................] - ETA: 1s - loss: 0.1224 - accuracy: 0.9629 110/938 [==>...........................] - ETA: 1s - loss: 0.1221 - accuracy: 0.9635 137/938 [===>..........................] - ETA: 1s - loss: 0.1237 - accuracy: 0.9629 164/938 [====>.........................] - ETA: 1s - loss: 0.1229 - accuracy: 0.9639 191/938 [=====>........................] - ETA: 1s - loss: 0.1213 - accuracy: 0.9636 218/938 [=====>........................] - ETA: 1s - loss: 0.1203 - accuracy: 0.9639 245/938 [======>.......................] - ETA: 1s - loss: 0.1212 - accuracy: 0.9638 272/938 [=======>......................] - ETA: 1s - loss: 0.1219 - accuracy: 0.9638 298/938 [========>.....................] - ETA: 1s - loss: 0.1205 - accuracy: 0.9641 324/938 [=========>....................] - ETA: 1s - loss: 0.1218 - accuracy: 0.9633 351/938 [==========>...................] - ETA: 1s - loss: 0.1225 - accuracy: 0.9625 378/938 [===========>..................] - ETA: 1s - loss: 0.1231 - accuracy: 0.9625 406/938 [===========>..................] - ETA: 1s - loss: 0.1238 - accuracy: 0.9626 433/938 [============>.................] - ETA: 0s - loss: 0.1249 - accuracy: 0.9625 461/938 [=============>................] - ETA: 0s - loss: 0.1238 - accuracy: 0.9626 487/938 [==============>...............] - ETA: 0s - loss: 0.1251 - accuracy: 0.9624 513/938 [===============>..............] - ETA: 0s - loss: 0.1248 - accuracy: 0.9625 539/938 [================>.............] - ETA: 0s - loss: 0.1237 - accuracy: 0.9627 565/938 [=================>............] - ETA: 0s - loss: 0.1225 - accuracy: 0.9632 592/938 [=================>............] - ETA: 0s - loss: 0.1233 - accuracy: 0.9630 619/938 [==================>...........] - ETA: 0s - loss: 0.1214 - accuracy: 0.9636 646/938 [===================>..........] - ETA: 0s - loss: 0.1212 - accuracy: 0.9637 674/938 [====================>.........] - ETA: 0s - loss: 0.1219 - accuracy: 0.9635 700/938 [=====================>........] - ETA: 0s - loss: 0.1221 - accuracy: 0.9634 726/938 [======================>.......] - ETA: 0s - loss: 0.1220 - accuracy: 0.9635 754/938 [=======================>......] - ETA: 0s - loss: 0.1221 - accuracy: 0.9634 781/938 [=======================>......] - ETA: 0s - loss: 0.1218 - accuracy: 0.9635 808/938 [========================>.....] - ETA: 0s - loss: 0.1217 - accuracy: 0.9637 837/938 [=========================>....] - ETA: 0s - loss: 0.1210 - accuracy: 0.9639 866/938 [==========================>...] - ETA: 0s - loss: 0.1208 - accuracy: 0.9641 894/938 [===========================>..] - ETA: 0s - loss: 0.1211 - accuracy: 0.9639 921/938 [============================>.] - ETA: 0s - loss: 0.1212 - accuracy: 0.9638 938/938 [==============================] - 2s 2ms/step - loss: 0.1212 - accuracy: 0.9638 - val_loss: 0.0947 - val_accuracy: 0.9726 Timed stopping at epoch 3 after training for 0:00:05 <keras.callbacks.History at 0x7f7a91776b90>