ตอบคำถาม BERT ด้วย TensorFlow Lite Model Maker

ดูบน TensorFlow.org ทำงานใน Google Colab ดูแหล่งที่มาบน GitHub ดาวน์โหลดโน๊ตบุ๊ค

ห้องสมุดชง TensorFlow Lite รุ่น ช่วยลดความยุ่งยากในการปรับตัวและการแปลงรูปแบบ TensorFlow เพื่อป้อนข้อมูลโดยเฉพาะอย่างยิ่งเมื่อมีการปรับใช้รูปแบบนี้ได้บนอุปกรณ์การใช้งานมิลลิลิตร

สมุดบันทึกนี้แสดงตัวอย่างแบบ end-to-end ที่ใช้ไลบรารี Model Maker เพื่อแสดงการดัดแปลงและการแปลงรูปแบบการตอบคำถามที่ใช้กันทั่วไปสำหรับงานตอบคำถาม

บทนำสู่งานตอบคำถามของ BERT

งานที่ได้รับการสนับสนุนในไลบรารีนี้เป็นงานตอบคำถามแบบแยกส่วน ซึ่งหมายความว่าเมื่อได้รับข้อความและคำถาม คำตอบคือช่วงของข้อความ ภาพด้านล่างแสดงตัวอย่างการตอบคำถาม

คำตอบอยู่ในช่วงทางเดิน (เครดิตภาพ: ทีมบล็อก )

สำหรับแบบจำลองของงานตอบคำถาม อินพุตควรเป็นข้อความและคู่คำถามที่ได้รับการประมวลผลล่วงหน้าแล้ว ผลลัพธ์ควรเป็นบันทึกเริ่มต้นและบันทึกสิ้นสุดสำหรับแต่ละโทเค็นในข้อความ ขนาดของอินพุตสามารถกำหนดและปรับเปลี่ยนได้ตามความยาวของข้อความและคำถาม

ภาพรวมตั้งแต่ต้นจนจบ

ข้อมูลโค้ดต่อไปนี้สาธิตวิธีรับโมเดลภายในโค้ดไม่กี่บรรทัด กระบวนการโดยรวมประกอบด้วย 5 ขั้นตอน: (1) เลือกโมเดล (2) โหลดข้อมูล (3) ฝึกโมเดลใหม่ (4) ประเมิน และ (5) ส่งออกไปยังรูปแบบ TensorFlow Lite

# Chooses a model specification that represents the model.
spec = model_spec.get('mobilebert_qa')

# Gets the training data and validation data.
train_data = DataLoader.from_squad(train_data_path, spec, is_training=True)
validation_data = DataLoader.from_squad(validation_data_path, spec, is_training=False)

# Fine-tunes the model.
model = question_answer.create(train_data, model_spec=spec)

# Gets the evaluation result.
metric = model.evaluate(validation_data)

# Exports the model to the TensorFlow Lite format with metadata in the export directory.
model.export(export_dir)

ส่วนต่อไปนี้จะอธิบายโค้ดโดยละเอียด

ข้อกำหนดเบื้องต้น

เมื่อต้องการเรียกใช้ตัวอย่างนี้ติดตั้งแพคเกจที่จำเป็นรวมทั้งแพคเกจเครื่องชงรุ่นจาก repo GitHub

pip install -q tflite-model-maker

นำเข้าแพ็คเกจที่จำเป็น

import numpy as np
import os

import tensorflow as tf
assert tf.__version__.startswith('2')

from tflite_model_maker import model_spec
from tflite_model_maker import question_answer
from tflite_model_maker.config import ExportFormat
from tflite_model_maker.question_answer import DataLoader
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/tensorflow_addons/utils/ensure_tf_install.py:67: UserWarning: Tensorflow Addons supports using Python ops for all Tensorflow versions above or equal to 2.3.0 and strictly below 2.6.0 (nightly versions are not supported). 
 The versions of TensorFlow you are currently using is 2.6.0 and is not supported. 
Some things might work, some things might not.
If you were to encounter a bug, do not file an issue.
If you want to make sure you're using a tested and supported configuration, either change the TensorFlow version or the TensorFlow Addons's version. 
You can find the compatibility matrix in TensorFlow Addon's readme:
https://github.com/tensorflow/addons
  UserWarning,
/tmpfs/src/tf_docs_env/lib/python3.7/site-packages/numba/core/errors.py:154: UserWarning: Insufficiently recent colorama version found. Numba requires colorama >= 0.3.9
  warnings.warn(msg)

"ภาพรวมตั้งแต่ต้นจนจบ" แสดงให้เห็นตัวอย่างง่ายๆ ตั้งแต่ต้นจนจบ ส่วนต่อไปนี้จะอธิบายทีละขั้นตอนเพื่อแสดงรายละเอียดเพิ่มเติม

เลือก model_spec ที่แสดงแบบจำลองสำหรับการตอบคำถาม

แต่ละ model_spec วัตถุหมายถึงรูปแบบที่เฉพาะเจาะจงสำหรับคำตอบของคำถาม ปัจจุบัน Model Maker รองรับรุ่น MobileBERT และ BERT-Base

รุ่นที่รองรับ ชื่อรุ่น_spec รายละเอียดรุ่น
โมบายเบิร์ต 'mobilebert_qa' เล็กกว่า 4.3 เท่า และเร็วกว่า BERT-Base 5.5 เท่า ในขณะที่บรรลุผลการแข่งขัน เหมาะสำหรับสถานการณ์บนอุปกรณ์
MobileBERT-SQUAD 'mobilebert_qa_squad' สถาปัตยกรรมรูปแบบเดียวกับรุ่น MobileBERT และรูปแบบการฝึกอบรมครั้งแรกแล้วใน SQuAD1.1
BERT-เบส 'เบิร์ต_กา' โมเดล BERT มาตรฐานที่ใช้กันอย่างแพร่หลายในงาน NLP

ในการกวดวิชานี้ MobileBERT-Squad ถูกนำมาใช้เป็นตัวอย่าง เนื่องจากรูปแบบการฝึกอบรมแล้วใน SQuAD1.1 ก็สามารถคุ้มครองได้เร็วขึ้นสำหรับคำถามคำตอบที่งาน

spec = model_spec.get('mobilebert_qa_squad')
2021-08-12 11:59:51.438945: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-12 11:59:51.447414: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-12 11:59:51.448405: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero

โหลดข้อมูลอินพุตเฉพาะสำหรับแอป ML บนอุปกรณ์และประมวลผลข้อมูลล่วงหน้า

TriviaQA เป็นชุดข้อมูลที่เข้าใจในการอ่านที่มีมากกว่า 650K อเนกประสงค์คำถามคำตอบหลักฐาน ในบทช่วยสอนนี้ คุณจะใช้ชุดย่อยของชุดข้อมูลนี้เพื่อเรียนรู้วิธีใช้ไลบรารี Model Maker

ในการโหลดข้อมูลแปลงชุด TriviaQA ไป SQuAD1.1 รูปแบบโดยใช้ แปลงสคริปต์ Python กับ --sample_size=8000 และชุดของ web ข้อมูล แก้ไขโค้ดการแปลงเล็กน้อยโดย:

  • ข้ามตัวอย่างที่ไม่พบคำตอบใด ๆ ในเอกสารบริบท
  • รับคำตอบเดิมในบริบทโดยไม่มีตัวพิมพ์ใหญ่หรือตัวพิมพ์เล็ก

ดาวน์โหลดเวอร์ชันเก็บถาวรของชุดข้อมูลที่แปลงแล้ว

train_data_path = tf.keras.utils.get_file(
    fname='triviaqa-web-train-8000.json',
    origin='https://storage.googleapis.com/download.tensorflow.org/models/tflite/dataset/triviaqa-web-train-8000.json')
validation_data_path = tf.keras.utils.get_file(
    fname='triviaqa-verified-web-dev.json',
    origin='https://storage.googleapis.com/download.tensorflow.org/models/tflite/dataset/triviaqa-verified-web-dev.json')
Downloading data from https://storage.googleapis.com/download.tensorflow.org/models/tflite/dataset/triviaqa-web-train-8000.json
32571392/32570663 [==============================] - 0s 0us/step
32579584/32570663 [==============================] - 0s 0us/step
Downloading data from https://storage.googleapis.com/download.tensorflow.org/models/tflite/dataset/triviaqa-verified-web-dev.json
1171456/1167744 [==============================] - 0s 0us/step
1179648/1167744 [==============================] - 0s 0us/step

คุณยังสามารถฝึกโมเดล MobileBERT ด้วยชุดข้อมูลของคุณเอง หากคุณกำลังใช้งานสมุดบันทึกนี้บน Colab ให้อัปโหลดข้อมูลของคุณโดยใช้แถบด้านข้างทางซ้าย

อัพโหลดไฟล์

หากคุณไม่ต้องการที่จะอัปโหลดข้อมูลของคุณไปยังเมฆคุณยังสามารถเรียกใช้ห้องสมุดแบบออฟไลน์โดยต่อไปนี้ คู่มือ

ใช้ DataLoader.from_squad วิธีการในการโหลดและ preprocess รูปแบบทีม ข้อมูลตามที่ระบุ model_spec คุณสามารถใช้รูปแบบ SQuAD2.0 หรือ SQuaAD1.1 ได้ การตั้งค่าพารามิเตอร์ version_2_with_negative เป็น True หมายถึงรูปแบบคือ SQuAD2.0 มิฉะนั้น รูปแบบจะเป็น SQUAAD1.1 โดยค่าเริ่มต้น version_2_with_negative เป็น False

train_data = DataLoader.from_squad(train_data_path, spec, is_training=True)
validation_data = DataLoader.from_squad(validation_data_path, spec, is_training=False)
2021-08-12 12:02:04.752380: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-08-12 12:02:04.753227: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-12 12:02:04.754341: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-12 12:02:04.755202: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-12 12:02:05.293390: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-12 12:02:05.294462: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-12 12:02:05.295445: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-08-12 12:02:05.296323: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1510] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 14648 MB memory:  -> device: 0, name: Tesla V100-SXM2-16GB, pci bus id: 0000:00:05.0, compute capability: 7.0

ปรับแต่งโมเดล TensorFlow

สร้างแบบจำลองการตอบคำถามแบบกำหนดเองตามข้อมูลที่โหลด create ฟังก์ชั่นประกอบด้วยขั้นตอนต่อไปนี้:

  1. สร้างแบบจำลองสำหรับคำตอบของคำถามที่ตาม model_spec
  2. ฝึกรูปแบบการตอบคำถาม epochs เริ่มต้นและขนาดชุดเริ่มต้นที่มีการตั้งค่าตามสองตัวแปร default_training_epochs และ default_batch_size ใน model_spec วัตถุ
model = question_answer.create(train_data, model_spec=spec)
INFO:tensorflow:Retraining the models...
INFO:tensorflow:Retraining the models...
2021-08-12 12:02:17.450548: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
Epoch 1/2
1067/1067 [==============================] - 423s 350ms/step - loss: 1.1346 - start_positions_loss: 1.1321 - end_positions_loss: 1.1371
Epoch 2/2
1067/1067 [==============================] - 373s 350ms/step - loss: 0.7933 - start_positions_loss: 0.7927 - end_positions_loss: 0.7939

ดูรายละเอียดโครงสร้างโมเดล

model.summary()
Model: "model"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_word_ids (InputLayer)     [(None, 384)]        0                                            
__________________________________________________________________________________________________
input_mask (InputLayer)         [(None, 384)]        0                                            
__________________________________________________________________________________________________
input_type_ids (InputLayer)     [(None, 384)]        0                                            
__________________________________________________________________________________________________
hub_keras_layer_v1v2 (HubKerasL {'start_logits': (No 24582914    input_word_ids[0][0]             
                                                                 input_mask[0][0]                 
                                                                 input_type_ids[0][0]             
__________________________________________________________________________________________________
start_positions (Lambda)        (None, None)         0           hub_keras_layer_v1v2[0][1]       
__________________________________________________________________________________________________
end_positions (Lambda)          (None, None)         0           hub_keras_layer_v1v2[0][0]       
==================================================================================================
Total params: 24,582,914
Trainable params: 24,582,914
Non-trainable params: 0
__________________________________________________________________________________________________

ประเมินแบบจำลองที่กำหนดเอง

ประเมินรูปแบบในการตรวจสอบข้อมูลและได้รับ Dict ของตัวชี้วัดรวมทั้ง f1 คะแนนและ exact match ฯลฯ โปรดทราบว่าตัวชี้วัดที่แตกต่างกันสำหรับ SQuAD1.1 และ SQuAD2.0

model.evaluate(validation_data)
INFO:tensorflow:Made predictions for 200 records.
INFO:tensorflow:Made predictions for 200 records.
INFO:tensorflow:Made predictions for 400 records.
INFO:tensorflow:Made predictions for 400 records.
INFO:tensorflow:Made predictions for 600 records.
INFO:tensorflow:Made predictions for 600 records.
INFO:tensorflow:Made predictions for 800 records.
INFO:tensorflow:Made predictions for 800 records.
INFO:tensorflow:Made predictions for 1000 records.
INFO:tensorflow:Made predictions for 1000 records.
INFO:tensorflow:Made predictions for 1200 records.
INFO:tensorflow:Made predictions for 1200 records.
{'exact_match': 0.5884353741496599, 'final_f1': 0.6621698029861295}

ส่งออกไปยังรุ่น TensorFlow Lite

แปลงรูปแบบการฝึกอบรมเพื่อจัดรูปแบบรูปแบบ TensorFlow Lite กับ เมตาดาต้า เพื่อให้คุณหลังจากนั้นสามารถใช้ในโปรแกรมประยุกต์ ML บนอุปกรณ์ ไฟล์คำศัพท์ถูกฝังอยู่ในข้อมูลเมตา ชื่อไฟล์ TFLite เริ่มต้นคือ model.tflite

ในแอปพลิเคชัน ML บนอุปกรณ์จำนวนมาก ขนาดโมเดลเป็นปัจจัยสำคัญ ดังนั้น ขอแนะนำให้คุณใช้ quantize model เพื่อทำให้เล็กลงและอาจทำงานได้เร็วขึ้น เทคนิคการหาปริมาณหลังการฝึกเริ่มต้นคือการหาปริมาณช่วงไดนามิกสำหรับรุ่น BERT และ MobileBERT

model.export(export_dir='.')
2021-08-12 12:16:05.811327: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
INFO:tensorflow:Assets written to: /tmp/tmp7t_bxd9h/saved_model/assets
INFO:tensorflow:Assets written to: /tmp/tmp7t_bxd9h/saved_model/assets
2021-08-12 12:16:35.499794: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:351] Ignored output_format.
2021-08-12 12:16:35.499841: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:354] Ignored drop_control_dependency.
2021-08-12 12:16:35.499849: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:360] Ignored change_concat_input_ranges.
2021-08-12 12:16:35.501017: I tensorflow/cc/saved_model/reader.cc:38] Reading SavedModel from: /tmp/tmp7t_bxd9h/saved_model
2021-08-12 12:16:35.567920: I tensorflow/cc/saved_model/reader.cc:90] Reading meta graph with tags { serve }
2021-08-12 12:16:35.567966: I tensorflow/cc/saved_model/reader.cc:132] Reading SavedModel debug info (if present) from: /tmp/tmp7t_bxd9h/saved_model
2021-08-12 12:16:35.922151: I tensorflow/cc/saved_model/loader.cc:211] Restoring SavedModel bundle.
2021-08-12 12:16:37.787828: I tensorflow/cc/saved_model/loader.cc:195] Running initialization op on SavedModel bundle at path: /tmp/tmp7t_bxd9h/saved_model
2021-08-12 12:16:38.783520: I tensorflow/cc/saved_model/loader.cc:283] SavedModel load for tags { serve }; Status: success: OK. Took 3282507 microseconds.
2021-08-12 12:16:40.489883: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:210] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.
2021-08-12 12:16:43.756590: I tensorflow/compiler/mlir/lite/flatbuffer_export.cc:1899] Estimated count of arithmetic ops: 18.380 G  ops, equivalently 9.190 G  MACs
2021-08-12 12:16:43.920701: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_0/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.920748: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_0/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.920754: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_0/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.920759: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_0/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.920765: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_0/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.920770: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_0/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.920775: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_0/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.920780: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_0/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.920797: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_1/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.920801: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_1/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.920806: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_1/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.920811: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_1/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.920817: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_1/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.920822: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_1/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.920826: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_1/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.920833: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_1/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.920848: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_2/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.920853: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_2/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.920858: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_2/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.920863: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_2/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.920870: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_2/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.920874: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_2/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.920879: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_2/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.920883: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_2/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.920897: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_3/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.920902: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_3/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.920907: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_3/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.920911: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_3/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.920917: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_3/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.920922: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_3/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.920926: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_3/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.920931: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_3/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.920949: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_4/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.920954: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_4/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.920958: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_4/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.920964: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_4/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.920970: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_4/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.920975: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_4/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.920980: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_4/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.920985: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_4/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.920999: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_5/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.921004: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_5/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.921008: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_5/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.921013: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_5/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.921020: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_5/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.921024: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_5/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.921029: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_5/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.921033: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_5/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.921048: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_6/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.921052: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_6/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.921057: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_6/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.921062: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_6/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.921068: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_6/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.921072: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_6/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.921077: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_6/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.921082: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_6/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.921096: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_7/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.921100: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_7/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.921105: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_7/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.921110: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_7/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.921116: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_7/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.921120: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_7/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.921125: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_7/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.921129: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_7/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.921146: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_8/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.921151: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_8/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.921155: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_8/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.921168: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_8/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.921174: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_8/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.921179: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_8/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.921183: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_8/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.921188: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_8/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.921201: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_9/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.921206: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_9/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.921211: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_9/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.921216: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_9/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.921222: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_9/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.921227: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_9/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.921232: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_9/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.921236: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_9/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.921250: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_10/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.921254: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_10/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.921259: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_10/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.921264: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_10/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.921270: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_10/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.921275: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_10/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.921280: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_10/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.921284: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_10/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.921298: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_11/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.921302: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_11/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.921307: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_11/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.921312: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_11/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.921318: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_11/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.921323: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_11/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.921327: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_11/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.921333: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_11/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.921348: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_12/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.921354: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_12/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.921359: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_12/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.921363: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_12/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.921370: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_12/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.921374: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_12/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.921379: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_12/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.921384: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_12/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.921398: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_13/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.921403: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_13/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.921408: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_13/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.921412: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_13/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.921418: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_13/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.921423: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_13/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.921428: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_13/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.921432: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_13/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.921446: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_14/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.921451: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_14/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.921456: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_14/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.921461: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_14/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.921467: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_14/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.921472: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_14/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.921477: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_14/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.921481: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_14/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.921507: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_15/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.921512: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_15/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.921517: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_15/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.921521: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_15/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.921527: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_15/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.921532: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_15/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.921537: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_15/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.921542: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_15/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.921556: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_16/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.921560: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_16/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.921565: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_16/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.921570: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_16/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.921576: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_16/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.921581: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_16/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.921585: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_16/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.921590: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_16/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.921604: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_17/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.921609: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_17/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.921613: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_17/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.921618: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_17/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.921624: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_17/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.921628: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_17/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.921633: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_17/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.921638: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_17/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.921651: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_18/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.921655: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_18/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.921660: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_18/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.921665: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_18/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.921672: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_18/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.921676: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_18/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.921681: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_18/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.921686: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_18/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.921699: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_19/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.921704: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_19/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.921708: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_19/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.921714: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_19/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.921720: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_19/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.921724: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_19/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.921729: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_19/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.921734: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_19/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.921747: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_20/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.921752: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_20/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.921756: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_20/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.921760: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_20/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.921766: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_20/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.921771: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_20/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.921776: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_20/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.921780: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_20/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.921795: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_21/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.921799: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_21/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.921804: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_21/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.921808: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_21/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.921815: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_21/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.921820: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_21/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.921824: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_21/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.921829: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_21/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.921843: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_22/attention/self/MatMul15 because it has no allocated buffer.
2021-08-12 12:16:43.921848: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_22/attention/self/MatMul17 because it has no allocated buffer.
2021-08-12 12:16:43.921853: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_22/attention/self/MatMul19 because it has no allocated buffer.
2021-08-12 12:16:43.921857: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_22/attention/self/MatMul21 because it has no allocated buffer.
2021-08-12 12:16:43.921863: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_22/attention/self/MatMul_114 because it has no allocated buffer.
2021-08-12 12:16:43.921868: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_22/attention/self/MatMul_116 because it has no allocated buffer.
2021-08-12 12:16:43.921872: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_22/attention/self/MatMul_118 because it has no allocated buffer.
2021-08-12 12:16:43.921877: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_22/attention/self/MatMul_120 because it has no allocated buffer.
2021-08-12 12:16:43.921890: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_23/attention/self/MatMul20 because it has no allocated buffer.
2021-08-12 12:16:43.921894: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_23/attention/self/MatMul22 because it has no allocated buffer.
2021-08-12 12:16:43.921899: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_23/attention/self/MatMul24 because it has no allocated buffer.
2021-08-12 12:16:43.921903: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_23/attention/self/MatMul26 because it has no allocated buffer.
2021-08-12 12:16:43.921909: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_23/attention/self/MatMul_125 because it has no allocated buffer.
2021-08-12 12:16:43.921914: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_23/attention/self/MatMul_127 because it has no allocated buffer.
2021-08-12 12:16:43.921918: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_23/attention/self/MatMul_129 because it has no allocated buffer.
2021-08-12 12:16:43.921923: I tensorflow/lite/tools/optimize/quantize_weights.cc:234] Skipping quantization of tensor bert/encoder/layer_23/attention/self/MatMul_131 because it has no allocated buffer.
INFO:tensorflow:Vocab file is inside the TFLite model with metadata.
INFO:tensorflow:Vocab file is inside the TFLite model with metadata.
INFO:tensorflow:Saved vocabulary in /tmp/tmpjncdf_eu/vocab.txt.
INFO:tensorflow:Saved vocabulary in /tmp/tmpjncdf_eu/vocab.txt.
INFO:tensorflow:Finished populating metadata and associated file to the model:
INFO:tensorflow:Finished populating metadata and associated file to the model:
INFO:tensorflow:./model.tflite
INFO:tensorflow:./model.tflite
INFO:tensorflow:The associated file that has been been packed to the model is:
INFO:tensorflow:The associated file that has been been packed to the model is:
INFO:tensorflow:['vocab.txt']
INFO:tensorflow:['vocab.txt']
INFO:tensorflow:TensorFlow Lite model exported successfully: ./model.tflite
INFO:tensorflow:TensorFlow Lite model exported successfully: ./model.tflite

คุณสามารถใช้ไฟล์รูปแบบ TensorFlow Lite ใน bert_qa ตรวจสอบการอ้างอิงโดยใช้ BertQuestionAnswerer API ใน TensorFlow Lite ห้องสมุดงาน โดยดาวน์โหลดได้จากแถบด้านซ้ายบน Colab

รูปแบบการส่งออกที่อนุญาตสามารถเป็นหนึ่งหรือรายการต่อไปนี้:

โดยค่าเริ่มต้น มันจะส่งออกโมเดล TensorFlow Lite พร้อมข้อมูลเมตาเท่านั้น คุณยังสามารถเลือกส่งออกไฟล์ต่างๆ ตัวอย่างเช่น การส่งออกเฉพาะไฟล์คำศัพท์ดังนี้:

model.export(export_dir='.', export_format=ExportFormat.VOCAB)
INFO:tensorflow:Saved vocabulary in ./vocab.txt.
INFO:tensorflow:Saved vocabulary in ./vocab.txt.

นอกจากนี้คุณยังสามารถประเมินรูปแบบ tflite กับ evaluate_tflite วิธี ขั้นตอนนี้คาดว่าจะใช้เวลานาน

model.evaluate_tflite('model.tflite', validation_data)
INFO:tensorflow:Made predictions for 100 records.
INFO:tensorflow:Made predictions for 100 records.
INFO:tensorflow:Made predictions for 200 records.
INFO:tensorflow:Made predictions for 200 records.
INFO:tensorflow:Made predictions for 300 records.
INFO:tensorflow:Made predictions for 300 records.
INFO:tensorflow:Made predictions for 400 records.
INFO:tensorflow:Made predictions for 400 records.
INFO:tensorflow:Made predictions for 500 records.
INFO:tensorflow:Made predictions for 500 records.
INFO:tensorflow:Made predictions for 600 records.
INFO:tensorflow:Made predictions for 600 records.
INFO:tensorflow:Made predictions for 700 records.
INFO:tensorflow:Made predictions for 700 records.
INFO:tensorflow:Made predictions for 800 records.
INFO:tensorflow:Made predictions for 800 records.
INFO:tensorflow:Made predictions for 900 records.
INFO:tensorflow:Made predictions for 900 records.
INFO:tensorflow:Made predictions for 1000 records.
INFO:tensorflow:Made predictions for 1000 records.
INFO:tensorflow:Made predictions for 1100 records.
INFO:tensorflow:Made predictions for 1100 records.
INFO:tensorflow:Made predictions for 1200 records.
INFO:tensorflow:Made predictions for 1200 records.
{'exact_match': 0.5918367346938775, 'final_f1': 0.6682598580557765}

การใช้งานขั้นสูง

create ฟังก์ชั่นเป็นส่วนหนึ่งที่สำคัญของห้องสมุดนี้ซึ่งใน model_spec พารามิเตอร์กำหนดสเปครุ่น BertQASpec ระดับปัจจุบันสนับสนุน มี 2 ​​รุ่น คือ รุ่น MobileBERT รุ่น BERT-Base create ฟังก์ชั่นประกอบด้วยขั้นตอนต่อไปนี้:

  1. สร้างแบบจำลองสำหรับคำตอบของคำถามที่ตาม model_spec
  2. ฝึกรูปแบบการตอบคำถาม

ส่วนนี้จะอธิบายหัวข้อขั้นสูงหลายหัวข้อ รวมถึงการปรับโมเดล การปรับพารามิเตอร์ไฮเปอร์พารามิเตอร์การฝึก ฯลฯ

ปรับรุ่น

คุณสามารถปรับโครงสร้างพื้นฐานรูปแบบเช่นพารามิเตอร์ seq_len และ query_len ใน BertQASpec ระดับ

พารามิเตอร์ที่ปรับได้สำหรับรุ่น:

  • seq_len : ความยาวของทางเดินที่จะป้อนเข้าสู่รูปแบบ
  • query_len : ความยาวของคำถามที่จะป้อนเข้าสู่รูปแบบ
  • doc_stride การก้าวย่างเมื่อทำวิธีการที่หน้าต่างบานเลื่อนที่จะใช้ชิ้นของเอกสาร
  • initializer_range : ในส่วนเบี่ยงเบนมาตรฐานของ truncated_normal_initializer สำหรับการเริ่มต้นทุกการฝึกอบรมน้ำหนัก
  • trainable : บูลีนไม่ว่าจะเป็นชั้นก่อนได้รับการฝึกฝนเป็นสุวินัย

พารามิเตอร์ที่ปรับได้สำหรับไปป์ไลน์การฝึกอบรม:

  • model_dir : ตำแหน่งของไฟล์รูปแบบด่าน หากไม่ได้ตั้งค่าไว้ ระบบจะใช้ไดเร็กทอรีชั่วคราว
  • dropout_rate : อัตราออกกลางคัน
  • learning_rate : อัตราการเรียนรู้เริ่มต้นสำหรับอดัม
  • predict_batch_size : ขนาดชุดการทำนาย
  • tpu : TPU อยู่เพื่อเชื่อมต่อกับ ใช้เฉพาะในกรณีที่ใช้ tpu

ตัวอย่างเช่น คุณสามารถฝึกโมเดลด้วยลำดับที่ยาวขึ้นได้ ถ้าคุณเปลี่ยนรูปแบบครั้งแรกที่คุณจะต้องสร้างใหม่ model_spec

new_spec = model_spec.get('mobilebert_qa')
new_spec.seq_len = 512

ขั้นตอนที่เหลือเหมือนกัน ทราบว่าคุณต้องเรียกทั้ง dataloader และ create ชิ้นส่วนที่เป็นรายละเอียดแบบที่แตกต่างอาจจะมีขั้นตอนก่อนการประมวลผลที่แตกต่างกัน

ปรับแต่งไฮเปอร์พารามิเตอร์การฝึก

คุณยังสามารถปรับแต่ง hyperparameters การฝึกอบรมเช่น epochs และ batch_size ส่งผลกระทบต่อประสิทธิภาพการทำงานรูปแบบ ตัวอย่างเช่น

  • epochs : ยุคสมัยมากขึ้นอาจเกิดประสิทธิภาพที่ดีขึ้น แต่อาจนำไปสู่ overfitting
  • batch_size : จำนวนตัวอย่างที่จะใช้ในขั้นตอนการฝึกอบรมหนึ่ง

ตัวอย่างเช่น คุณสามารถฝึกด้วยช่วงเวลาที่มากขึ้นและด้วยขนาดแบทช์ที่ใหญ่กว่า เช่น:

model = question_answer.create(train_data, model_spec=spec, epochs=5, batch_size=64)

เปลี่ยนรูปแบบสถาปัตยกรรม

คุณสามารถเปลี่ยนฐานรถไฟจำลองข้อมูลของคุณในโดยการเปลี่ยน model_spec ตัวอย่างเช่น หากต้องการเปลี่ยนเป็นโมเดล BERT-Base ให้รัน:

spec = model_spec.get('bert_qa')

ขั้นตอนที่เหลือเหมือนกัน

ปรับแต่งการหาปริมาณหลังการฝึกในรุ่น TensorFlow Lite

โพสต์การฝึกอบรม quantization เป็นเทคนิคการแปลงที่สามารถลดขนาดรูปแบบและแฝงอนุมานในขณะที่ยังมีการปรับปรุงความเร็วของ CPU และการเร่งความเร็วฮาร์ดแวร์อนุมานกับการย่อยสลายเล็ก ๆ น้อย ๆ ในความถูกต้องรูปแบบ ดังนั้นจึงใช้กันอย่างแพร่หลายในการปรับโมเดลให้เหมาะสม

ไลบรารี Model Maker จะใช้เทคนิคการวัดค่าหลังการฝึกเริ่มต้นเมื่อส่งออกโมเดล หากคุณต้องการที่จะปรับแต่งควอนโพสต์การฝึกอบรมรุ่นชงสนับสนุนตัวเลือกการโพสต์การฝึกอบรม quantization ต่างๆโดยใช้ QuantizationConfig เช่นกัน ลองใช้การควอนไทซ์ float16 เป็นตัวอย่าง ขั้นแรก กำหนดการกำหนดค่า quantization

config = QuantizationConfig.for_float16()

จากนั้นเราจะส่งออกโมเดล TensorFlow Lite ด้วยการกำหนดค่าดังกล่าว

model.export(export_dir='.', tflite_filename='model_fp16.tflite', quantization_config=config)

อ่านเพิ่มเติม

คุณสามารถอ่านของเรา BERT คำถามและคำตอบ เช่นการเรียนรู้รายละเอียดทางเทคนิค สำหรับข้อมูลเพิ่มเติม โปรดดูที่: