|---|---|---|---| | [View on TensorFlow.org](https://www.tensorflow.org/text/guide/text_tf_lite) | [Run in Google Colab](https://colab.research.google.com/github/tensorflow/text/blob/master/docs/guide/text_tf_lite.ipynb) | [View on GitHub](https://github.com/tensorflow/text/blob/master/docs/guide/text_tf_lite.ipynb) | [Download notebook](https://storage.googleapis.com/tensorflow_docs/text/docs/guide/text_tf_lite.ipynb) | ## Overview Machine learning models are frequently deployed using TensorFlow Lite to mobile, embedded, and IoT devices to improve data privacy and lower response times. These models often require support for text processing operations. TensorFlow Text version 2.7 and higher provides improved performance, reduced binary sizes, and operations specifically optimized for use in these environments. ## Text operators The following TensorFlow Text classes and functions can be used from within a TensorFlow Lite model. - [`text.ByteSplitter`](https://www.tensorflow.org/text/api_docs/python/text/ByteSplitter) - [`text.FastBertNormalizer`](https://www.tensorflow.org/text/api_docs/python/text/FastBertNormalizer) - [`text.FastBertTokenizer`](https://www.tensorflow.org/text/api_docs/python/text/FastBertTokenizer) - [`text.FastWordpieceTokenizer`](https://www.tensorflow.org/text/api_docs/python/text/FastWordpieceTokenizer) - [`text.FastSentencepieceTokenizer`](https://www.tensorflow.org/text/api_docs/python/text/FastSentencepieceTokenizer) - [`text.WhitespaceTokenizer`](https://www.tensorflow.org/text/api_docs/python/text/WhitespaceTokenizer) - [`text.ngrams`](https://www.tensorflow.org/text/api_docs/python/text/ngrams) ## Model Example pip install -q -U tensorflow-text tf-keras import os os.environ["TF_USE_LEGACY_KERAS"] = "1" from absl import app import numpy as np import tensorflow as tf import tensorflow_text as tf_text from tensorflow.lite.python import interpreter The following code example shows the conversion process and interpretation in Python using a simple test model. Note that the output of a model cannot be a [`tf.RaggedTensor`](https://www.tensorflow.org/api_docs/python/tf/RaggedTensor) object when you are using TensorFlow Lite. However, you can return the components of a [`tf.RaggedTensor`](https://www.tensorflow.org/api_docs/python/tf/RaggedTensor) object or convert it using its `to_tensor` function. See [the RaggedTensor guide](https://www.tensorflow.org/guide/ragged_tensor) for more details. class TokenizerModel(tf.keras.Model): def __init__(self, **kwargs): super().__init__(**kwargs) self.tokenizer = tf_text.WhitespaceTokenizer() @tf.function(input_signature=[ tf.TensorSpec(shape=[None], dtype=tf.string, name='input') ]) def call(self, input_tensor): return { 'tokens': self.tokenizer.tokenize(input_tensor).flat_values } # Test input data. input_data = np.array(['Some minds are better kept apart']) # Define a Keras model. model = TokenizerModel() # Perform TensorFlow Text inference. tf_result = model(tf.constant(input_data)) print('TensorFlow result = ', tf_result['tokens']) ``` TensorFlow result = tf.Tensor([b'Some' b'minds' b'are' b'better' b'kept' b'apart'], shape=(6,), dtype=string) ``` ## Convert the TensorFlow model to TensorFlow Lite When converting a TensorFlow model with TensorFlow Text operators to TensorFlow Lite, you need to indicate to the `TFLiteConverter` that there are custom operators using the `allow_custom_ops` attribute as in the example below. You can then run the model conversion as you normally would. Review the [TensorFlow Lite converter](https://www.tensorflow.org/lite/convert) documentation for a detailed guide on the basics of model conversion. # Convert to TensorFlow Lite. converter = tf.lite.TFLiteConverter.from_keras_model(model) converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS] converter.allow_custom_ops = True tflite_model = converter.convert() ``` INFO:tensorflow:Assets written to: /tmpfs/tmp/tmpwayy3_hl/assets INFO:tensorflow:Assets written to: /tmpfs/tmp/tmpwayy3_hl/assets WARNING: All log messages before absl::InitializeLog() is called are written to STDERR W0000 00:00:1793452737.230138 97390 tf_tfl_flatbuffer_helpers.cc:364] Ignored output_format. W0000 00:00:1793452737.230186 97390 tf_tfl_flatbuffer_helpers.cc:367] Ignored drop_control_dependency. W0000 00:00:1793452737.397290 97390 flatbuffer_export.cc:4248] The following operation(s) need TFLite custom op implementation(s): Custom ops: TFText>WhitespaceTokenizeWithOffsetsV2 Details: tf.TFText>WhitespaceTokenizeWithOffsetsV2(tensor, tensor) -> (tensor, tensor, tensor, tensor) : {device = ""} See instructions: https://www.tensorflow.org/lite/guide/ops_custom ``` ## Inference For the TensorFlow Lite interpreter to properly read your model containing TensorFlow Text operators, you must configure it to use these custom operators, and provide registration methods for them. Use `tf_text.tflite_registrar.SELECT_TFTEXT_OPS` to provide the full suite of registration functions for the supported TensorFlow Text operators to `InterpreterWithCustomOps`. Note, that while the example below shows inference in Python, the steps are similar in other languages with some minor API translations, and the necessity to build the `tflite_registrar` into your binary. See [TensorFlow Lite Inference](https://www.tensorflow.org/lite/guide/inference) for more details. # Perform TensorFlow Lite inference. interp = interpreter.InterpreterWithCustomOps( model_content=tflite_model, custom_op_registerers=tf_text.tflite_registrar.SELECT_TFTEXT_OPS) interp.get_signature_list() ``` /tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow/lite/python/interpreter.py:457: UserWarning: Warning: tf.lite.Interpreter is deprecated and is scheduled for deletion in TF 2.20. Please use the LiteRT interpreter from the ai_edge_litert package. See the [migration guide](https://ai.google.dev/edge/litert/migration) for details. warnings.warn(_INTERPRETER_DELETION_WARNING) {'serving_default': {'inputs': ['input'], 'outputs': ['tokens']} } ``` Next, the TensorFlow Lite interpreter is invoked with the input, providing a result which matches the TensorFlow result from above. tokenize = interp.get_signature_runner('serving_default') output = tokenize(input=input_data) print('TensorFlow Lite result = ', output['tokens']) ``` TensorFlow Lite result = [b'Some' b'minds' b'are' b'better' b'kept' b'apart'] INFO: Created TensorFlow Lite XNNPACK delegate for CPU. ```