Google I/O is a wrap! Catch up on TensorFlow sessions View sessions

tflite_support.metadata_writers.metadata_info.SentencePieceTokenizerMd

A container for the sentence piece tokenizer 1 metadata information.

sentence_piece_model_path path to the sentence piece model file.
vocab_file_path path to the vocabulary file.

Methods

create_metadata

View source

Creates the sentence piece tokenizer metadata based on the information.

Returns
A Flatbuffers Python object of the sentence piece tokenizer metadata.