Base DataLoader class for Searcher task.

embedder_path Path to the TFLite Embedder model file.
dataset Embedding dataset used to build on-device ScaNN index file. The dataset shape should be (dataset_size, embedding_dim). If None, dataset will be generated from raw input data later.
metadata The metadata for each data in the dataset. The length of metadata should be same as dataset and passed in the same order as dataset. If dataset is set, metadata should be set as well.

dataset Gets the dataset.

Due to performance consideration, we don't return a copy, but the returned self._dataset should never be changed.

embedder_path Gets the path to the TFLite Embedder model file.
metadata Gets the metadata.



View source

Appends the dataset.

Don't check if embedders from the two data loader are the same in this function. Users are responsible to keep the embedder identical.

data_loader The data loader in which the data will be appended.


View source