tflite_model_maker.searcher.TextDataLoader

DataLoader class for Text Searcher.

Inherits From: DataLoader

Used in the notebooks

Used in the tutorials

embedder Embedder to generate embedding from raw input image.

dataset Gets the dataset.

Due to performance consideration, we don't return a copy, but the returned self._dataset should never be changed.

embedder_path Gets the path to the TFLite Embedder model file.
metadata Gets the metadata.

Methods

append

View source

Appends the dataset.

Don't check if embedders from the two data loader are the same in this function. Users are responsible to keep the embedder identical.

Args
data_loader The data loader in which the data will be appended.

create

View source

Creates DataLoader for the Text Searcher task.

Args
text_embedder_path Path to the ".tflite" text embedder model. case and L2 norm is thus achieved through TF Lite inference.
l2_normalize Whether to normalize the returned feature vector with L2 norm. Use this option only if the model does not already contain a native L2_NORMALIZATION TF Lite Op. In most cases, this is already the case and L2 norm is thus achieved through TF Lite inference.

Returns
DataLoader object created for the Text Searcher task.

load_from_csv

View source

Loads text data from csv file that includes a "header" line with titles.

Users can load text from different csv files one by one. For instance,

# Creates data_loader instance.
data_loader = text_searcher_dataloader.DataLoader.create(tflite_path)

# Loads text, first from `text_path1` and secondly from `text_path2`.
data_loader.load_from_csv(
    text_path1, text_column='text', metadata_column='metadata')
data_loader.load_from_csv(
    text_path2, text_column='text', metadata_column='metadata')

Args
path Text csv file path to be loaded.
text_column Column name for input text.
metadata_column Column name for user metadata associated with each input text.
delimiter Character used to separate fields.
quotechar Character used to quote fields containing special characters.

__len__

View source