tflite_support.metadata_writers.writer_utils.get_tokenizer_associated_files

Gets a list of associated files packed in the tokenzier_options.

tokenizer_options a tokenizer metadata object. Support the following tokenizer types:

  1. BertTokenizerOptions: https://github.com/tensorflow/tflite-support/blob/b80289c4cd1224d0e1836c7654e82f070f9eefaa/tensorflow_lite_support/metadata/metadata_schema.fbs#L436
  2. SentencePieceTokenizerOptions: https://github.com/tensorflow/tflite-support/blob/b80289c4cd1224d0e1836c7654e82f070f9eefaa/tensorflow_lite_support/metadata/metadata_schema.fbs#L473
  3. RegexTokenizerOptions: https://github.com/tensorflow/tflite-support/blob/b80289c4cd1224d0e1836c7654e82f070f9eefaa/tensorflow_lite_support/metadata/metadata_schema.fbs#L475

A list of associated files included in tokenizer_options.