 TensorFlow.org에서 보기 |
 Run in Google Colab |
 GitHub에서 소스 보기 |
 노트북 다운로드하기 |
개요
양자화 인식 훈련의 엔드 투 엔드 예제를 시작합니다.
기타 페이지
양자화 인식 훈련이 무엇인지에 대한 소개와 이를 사용해야 하는지에 대한 결정(지원되는 내용 포함)은 개요 페이지를 참조하세요.
사용 사례에 필요한 API를 빠르게 찾으려면(8bit로 모델을 완전히 양자화하는 것 이상), 종합 가이드를 참조하세요.
요약
이 튜토리얼에서는 다음을 수행합니다.
- MNIST용
tf.keras모델을 처음부터 훈련합니다. - 양자화 인식 학습 API를 적용하여 모델을 미세 조정하고, 정확성을 확인하고, 양자화 인식 모델을 내보냅니다.
- 모델을 사용하여 TFLite 백엔드에 대해 실제로 양자화된 모델을 만듭니다.
- TFLite와 4배 더 작아진 모델에서 정확성의 지속성을 확인합니다. 모바일에서의 지연 시간 이점을 확인하려면, TFLite 앱 리포지토리에서 TFLite 예제를 사용해 보세요.
설정
pip uninstall -y tensorflowpip install -q tf-nightlypip install -q tensorflow-model-optimization
import tempfile
import os
import tensorflow as tf
from tensorflow import keras
양자화 인식 훈련 없이 MNIST 모델 훈련하기
# Load MNIST dataset
mnist = keras.datasets.mnist
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()
# Normalize the input image so that each pixel value is between 0 to 1.
train_images = train_images / 255.0
test_images = test_images / 255.0
# Define the model architecture.
model = keras.Sequential([
keras.layers.InputLayer(input_shape=(28, 28)),
keras.layers.Reshape(target_shape=(28, 28, 1)),
keras.layers.Conv2D(filters=12, kernel_size=(3, 3), activation='relu'),
keras.layers.MaxPooling2D(pool_size=(2, 2)),
keras.layers.Flatten(),
keras.layers.Dense(10)
])
# Train the digit classification model
model.compile(optimizer='adam',
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
model.fit(
train_images,
train_labels,
epochs=1,
validation_split=0.1,
)
1/1688 [..............................] - ETA: 1:01:25 - loss: 2.2936 - accuracy: 0.0625 22/1688 [..............................] - ETA: 4s - loss: 2.1873 - accuracy: 0.2267 45/1688 [..............................] - ETA: 3s - loss: 2.0397 - accuracy: 0.3321 68/1688 [>.............................] - ETA: 3s - loss: 1.8877 - accuracy: 0.4111 91/1688 [>.............................] - ETA: 3s - loss: 1.7481 - accuracy: 0.4695 114/1688 [=>............................] - ETA: 3s - loss: 1.6303 - accuracy: 0.5135 138/1688 [=>............................] - ETA: 3s - loss: 1.5270 - accuracy: 0.5492 162/1688 [=>............................] - ETA: 3s - loss: 1.4395 - accuracy: 0.5782 185/1688 [==>...........................] - ETA: 3s - loss: 1.3690 - accuracy: 0.6011 208/1688 [==>...........................] - ETA: 3s - loss: 1.3080 - accuracy: 0.6204 231/1688 [===>..........................] - ETA: 3s - loss: 1.2543 - accuracy: 0.6371 255/1688 [===>..........................] - ETA: 3s - loss: 1.2049 - accuracy: 0.6523 279/1688 [===>..........................] - ETA: 3s - loss: 1.1607 - accuracy: 0.6657 302/1688 [====>.........................] - ETA: 3s - loss: 1.1229 - accuracy: 0.6771 326/1688 [====>.........................] - ETA: 2s - loss: 1.0874 - accuracy: 0.6876 349/1688 [=====>........................] - ETA: 2s - loss: 1.0565 - accuracy: 0.6968 372/1688 [=====>........................] - ETA: 2s - loss: 1.0280 - accuracy: 0.7053 394/1688 [======>.......................] - ETA: 2s - loss: 1.0030 - accuracy: 0.7127 418/1688 [======>.......................] - ETA: 2s - loss: 0.9779 - accuracy: 0.7201 441/1688 [======>.......................] - ETA: 2s - loss: 0.9557 - accuracy: 0.7268 465/1688 [=======>......................] - ETA: 2s - loss: 0.9341 - accuracy: 0.7331 489/1688 [=======>......................] - ETA: 2s - loss: 0.9140 - accuracy: 0.7391 513/1688 [========>.....................] - ETA: 2s - loss: 0.8951 - accuracy: 0.7446 536/1688 [========>.....................] - ETA: 2s - loss: 0.8782 - accuracy: 0.7496 559/1688 [========>.....................] - ETA: 2s - loss: 0.8622 - accuracy: 0.7543 583/1688 [=========>....................] - ETA: 2s - loss: 0.8465 - accuracy: 0.7589 607/1688 [=========>....................] - ETA: 2s - loss: 0.8316 - accuracy: 0.7632 630/1688 [==========>...................] - ETA: 2s - loss: 0.8182 - accuracy: 0.7671 654/1688 [==========>...................] - ETA: 2s - loss: 0.8049 - accuracy: 0.7709 677/1688 [===========>..................] - ETA: 2s - loss: 0.7927 - accuracy: 0.7744 700/1688 [===========>..................] - ETA: 2s - loss: 0.7812 - accuracy: 0.7778 723/1688 [===========>..................] - ETA: 2s - loss: 0.7701 - accuracy: 0.7809 746/1688 [============>.................] - ETA: 2s - loss: 0.7595 - accuracy: 0.7840 769/1688 [============>.................] - ETA: 2s - loss: 0.7494 - accuracy: 0.7869 792/1688 [=============>................] - ETA: 1s - loss: 0.7398 - accuracy: 0.7896 815/1688 [=============>................] - ETA: 1s - loss: 0.7304 - accuracy: 0.7923 839/1688 [=============>................] - ETA: 1s - loss: 0.7211 - accuracy: 0.7950 862/1688 [==============>...............] - ETA: 1s - loss: 0.7125 - accuracy: 0.7974 885/1688 [==============>...............] - ETA: 1s - loss: 0.7042 - accuracy: 0.7998 908/1688 [===============>..............] - ETA: 1s - loss: 0.6962 - accuracy: 0.8021 931/1688 [===============>..............] - ETA: 1s - loss: 0.6886 - accuracy: 0.8043 954/1688 [===============>..............] - ETA: 1s - loss: 0.6811 - accuracy: 0.8064 978/1688 [================>.............] - ETA: 1s - loss: 0.6736 - accuracy: 0.8085 1001/1688 [================>.............] - ETA: 1s - loss: 0.6667 - accuracy: 0.8105 1025/1688 [=================>............] - ETA: 1s - loss: 0.6597 - accuracy: 0.8125 1048/1688 [=================>............] - ETA: 1s - loss: 0.6532 - accuracy: 0.8143 1072/1688 [==================>...........] - ETA: 1s - loss: 0.6467 - accuracy: 0.8162 1097/1688 [==================>...........] - ETA: 1s - loss: 0.6401 - accuracy: 0.8181 1121/1688 [==================>...........] - ETA: 1s - loss: 0.6339 - accuracy: 0.8198 1143/1688 [===================>..........] - ETA: 1s - loss: 0.6284 - accuracy: 0.8214 1166/1688 [===================>..........] - ETA: 1s - loss: 0.6229 - accuracy: 0.8230 1189/1688 [====================>.........] - ETA: 1s - loss: 0.6175 - accuracy: 0.8245 1212/1688 [====================>.........] - ETA: 1s - loss: 0.6122 - accuracy: 0.8260 1235/1688 [====================>.........] - ETA: 0s - loss: 0.6070 - accuracy: 0.8275 1259/1688 [=====================>........] - ETA: 0s - loss: 0.6018 - accuracy: 0.8289 1282/1688 [=====================>........] - ETA: 0s - loss: 0.5970 - accuracy: 0.8303 1306/1688 [======================>.......] - ETA: 0s - loss: 0.5920 - accuracy: 0.8317 1330/1688 [======================>.......] - ETA: 0s - loss: 0.5872 - accuracy: 0.8331 1353/1688 [=======================>......] - ETA: 0s - loss: 0.5827 - accuracy: 0.8344 1376/1688 [=======================>......] - ETA: 0s - loss: 0.5783 - accuracy: 0.8356 1399/1688 [=======================>......] - ETA: 0s - loss: 0.5740 - accuracy: 0.8368 1423/1688 [========================>.....] - ETA: 0s - loss: 0.5697 - accuracy: 0.8381 1447/1688 [========================>.....] - ETA: 0s - loss: 0.5654 - accuracy: 0.8393 1471/1688 [=========================>....] - ETA: 0s - loss: 0.5612 - accuracy: 0.8405 1496/1688 [=========================>....] - ETA: 0s - loss: 0.5569 - accuracy: 0.8417 1521/1688 [==========================>...] - ETA: 0s - loss: 0.5527 - accuracy: 0.8429 1543/1688 [==========================>...] - ETA: 0s - loss: 0.5491 - accuracy: 0.8439 1567/1688 [==========================>...] - ETA: 0s - loss: 0.5453 - accuracy: 0.8450 1590/1688 [===========================>..] - ETA: 0s - loss: 0.5417 - accuracy: 0.8460 1614/1688 [===========================>..] - ETA: 0s - loss: 0.5380 - accuracy: 0.8470 1637/1688 [============================>.] - ETA: 0s - loss: 0.5346 - accuracy: 0.8480 1661/1688 [============================>.] - ETA: 0s - loss: 0.5310 - accuracy: 0.8490 1684/1688 [============================>.] - ETA: 0s - loss: 0.5277 - accuracy: 0.8499 1688/1688 [==============================] - 7s 3ms/step - loss: 0.5270 - accuracy: 0.8501 - val_loss: 0.1138 - val_accuracy: 0.9703 <tensorflow.python.keras.callbacks.History at 0x7f9b24977550>
양자화 인식 훈련으로 사전 훈련된 모델 복제 및 미세 조정
모델 정의하기
전체 모델에 양자화 인식 훈련을 적용하고 모델 요약에서 이를 확인합니다. 이제 모든 레이어 앞에 "quant"가 붙습니다.
결과 모델은 양자화를 인식하지만, 양자화되지는 않습니다(예: 가중치가 int8 대신 float32임). 다음 섹션에서는 양자화 인식 모델에서 양자화된 모델을 만드는 방법을 보여줍니다.
종합 가이드에서 모델 정확성의 향상을 위해 일부 레이어를 양자화하는 방법을 볼 수 있습니다.
import tensorflow_model_optimization as tfmot
quantize_model = tfmot.quantization.keras.quantize_model
# q_aware stands for for quantization aware.
q_aware_model = quantize_model(model)
# `quantize_model` requires a recompile.
q_aware_model.compile(optimizer='adam',
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
q_aware_model.summary()
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= quantize_layer (QuantizeLaye (None, 28, 28) 3 _________________________________________________________________ quant_reshape (QuantizeWrapp (None, 28, 28, 1) 1 _________________________________________________________________ quant_conv2d (QuantizeWrappe (None, 26, 26, 12) 147 _________________________________________________________________ quant_max_pooling2d (Quantiz (None, 13, 13, 12) 1 _________________________________________________________________ quant_flatten (QuantizeWrapp (None, 2028) 1 _________________________________________________________________ quant_dense (QuantizeWrapper (None, 10) 20295 ================================================================= Total params: 20,448 Trainable params: 20,410 Non-trainable params: 38 _________________________________________________________________
기준선과 비교하여 모델 훈련 및 평가하기
하나의 epoch 동안 모델을 훈련한 후 미세 조정을 시연하려면 훈련 데이터의 하위 집합에 대한 양자화 인식 훈련으로 미세 조정합니다.
train_images_subset = train_images[0:1000] # out of 60000
train_labels_subset = train_labels[0:1000]
q_aware_model.fit(train_images_subset, train_labels_subset,
batch_size=500, epochs=1, validation_split=0.1)
1/2 [==============>...............] - ETA: 0s - loss: 0.1233 - accuracy: 0.9660 2/2 [==============================] - 1s 248ms/step - loss: 0.1391 - accuracy: 0.9613 - val_loss: 0.1508 - val_accuracy: 0.9700 <tensorflow.python.keras.callbacks.History at 0x7f9bcc57c128>
이 예제의 경우, 기준선과 비교하여 양자화 인식 훈련 후 테스트 정확성의 손실이 거의 없습니다.
_, baseline_model_accuracy = model.evaluate(
test_images, test_labels, verbose=0)
_, q_aware_model_accuracy = q_aware_model.evaluate(
test_images, test_labels, verbose=0)
print('Baseline test accuracy:', baseline_model_accuracy)
print('Quant test accuracy:', q_aware_model_accuracy)
Baseline test accuracy: 0.9616000056266785 Quant test accuracy: 0.9625999927520752
TFLite 백엔드를 위한 양자화 모델 생성하기
다음을 통해 int8 가중치 및 uint8 활성화를 사용하여 실제로 양자화된 모델을 얻게 됩니다.
converter = tf.lite.TFLiteConverter.from_keras_model(q_aware_model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
quantized_tflite_model = converter.convert()
WARNING:absl:Found untraced functions such as reshape_layer_call_and_return_conditional_losses, reshape_layer_call_fn, conv2d_layer_call_and_return_conditional_losses, conv2d_layer_call_fn, max_pooling2d_layer_call_and_return_conditional_losses while saving (showing 5 of 25). These functions will not be directly callable after loading. INFO:tensorflow:Assets written to: /tmp/tmpmfpzobkx/assets INFO:tensorflow:Assets written to: /tmp/tmpmfpzobkx/assets
TF에서 TFLite까지 정확성의 지속성 확인하기
테스트 데이터세트에 대해 TF Lite 모델을 평가하는 도우미 함수를 정의합니다.
import numpy as np
def evaluate_model(interpreter):
input_index = interpreter.get_input_details()[0]["index"]
output_index = interpreter.get_output_details()[0]["index"]
# Run predictions on every image in the "test" dataset.
prediction_digits = []
for i, test_image in enumerate(test_images):
if i % 1000 == 0:
print('Evaluated on {n} results so far.'.format(n=i))
# Pre-processing: add batch dimension and convert to float32 to match with
# the model's input data format.
test_image = np.expand_dims(test_image, axis=0).astype(np.float32)
interpreter.set_tensor(input_index, test_image)
# Run inference.
interpreter.invoke()
# Post-processing: remove batch dimension and find the digit with highest
# probability.
output = interpreter.tensor(output_index)
digit = np.argmax(output()[0])
prediction_digits.append(digit)
print('\n')
# Compare prediction results with ground truth labels to calculate accuracy.
prediction_digits = np.array(prediction_digits)
accuracy = (prediction_digits == test_labels).mean()
return accuracy
양자화 모델을 평가하고 TensorFlow의 정확성이 TFLite 백엔드까지 유지되는지 확인합니다.
interpreter = tf.lite.Interpreter(model_content=quantized_tflite_model)
interpreter.allocate_tensors()
test_accuracy = evaluate_model(interpreter)
print('Quant TFLite test_accuracy:', test_accuracy)
print('Quant TF test accuracy:', q_aware_model_accuracy)
Evaluated on 0 results so far. Evaluated on 1000 results so far. Evaluated on 2000 results so far. Evaluated on 3000 results so far. Evaluated on 4000 results so far. Evaluated on 5000 results so far. Evaluated on 6000 results so far. Evaluated on 7000 results so far. Evaluated on 8000 results so far. Evaluated on 9000 results so far. Quant TFLite test_accuracy: 0.9626 Quant TF test accuracy: 0.9625999927520752
양자화로 4배 더 작아진 모델 확인하기
float TFLite 모델을 생성한 다음 TFLite 양자화 모델이 4배 더 작아진 것을 확인합니다.
# Create float TFLite model.
float_converter = tf.lite.TFLiteConverter.from_keras_model(model)
float_tflite_model = float_converter.convert()
# Measure sizes of models.
_, float_file = tempfile.mkstemp('.tflite')
_, quant_file = tempfile.mkstemp('.tflite')
with open(quant_file, 'wb') as f:
f.write(quantized_tflite_model)
with open(float_file, 'wb') as f:
f.write(float_tflite_model)
print("Float model in Mb:", os.path.getsize(float_file) / float(2**20))
print("Quantized model in Mb:", os.path.getsize(quant_file) / float(2**20))
INFO:tensorflow:Assets written to: /tmp/tmpsmkiqoq3/assets INFO:tensorflow:Assets written to: /tmp/tmpsmkiqoq3/assets Float model in Mb: 0.08058547973632812 Quantized model in Mb: 0.0234527587890625
결론
이 튜토리얼에서는 TensorFlow Model Optimization Toolkit API를 사용하여 양자화 인식 모델을 만든 다음 TFLite 백엔드용 양자화 모델을 만드는 방법을 살펴보았습니다.
정확성 차이를 최소화하면서 MNIST 모델의 크기를 4배 압축하는 이점을 확인했습니다. 모바일에서의 지연 시간 이점을 확인하려면, TFLite 앱 리포지토리에서 TFLite 예제를 사용해 보세요.
이 새로운 기능은 리소스가 제한된 환경에서 배포할 때 특히 중요하므로 사용해 볼 것을 권장합니다.