 View on TensorFlow.org
|
 Run in Google Colab
|
 View source on GitHub
|
 Download notebook
|
TensorFlow Ranking can handle heterogeneous dense and sparse features, and scales up to millions of data points. However, building and deploying a learning to rank model to operate at scale creates additional challenges beyond simply designing a model. The Ranking library provides workflow utility classes for building distributed training for large-scale ranking applications. For more information about these features, see the TensorFlow Ranking Overview.
This tutorial shows you how to build a ranking model that uses BERT for scoring. BERT is a highly effective pretrained module to effective encode textual features into contextualized word embeddings. We use BERT to initialize the ranking model and finetune the model using a ranking loss.
ANTIQUE dataset
In this tutorial, you will build a ranking model for ANTIQUE, a question-answering dataset using BERT as the scoring function. Bidirectional Encoder Representations from Transformers (BERT) is a transformer-based machine learning technique which has proven to be effective in many natural language processing (NLP) tasks. Recent work on TFR-BERT has shown BERT to be an effective scoring function for learning-to-rank tasks.
Given a query, and a list of answers, the objective of the ranking model is to rank the answers with optimal rank related metrics, such as NDCG. For more details about ranking metrics, review evaluation measures offline metrics.
ANTIQUE is a publicly available dataset for open-domain non-factoid question answering, collected from Yahoo! answers. Each question has a list of answers, whose relevance are graded on a scale of 0-4, 0 for irrelevant and 4 for fully relevant. The list size can vary depending on the query, so we use a fixed "list size" of 50, where the list is either truncated or padded with default values. The dataset is split into 2206 queries for training and 200 queries for testing. For more details, please read the technical paper on arXiv.
Setup
Download and install the TensorFlow Ranking and TensorFlow Model Garden packages.
pip install -q tensorflow-ranking tf-models-officialImport TensorFlow Ranking and useful libraries through the notebook.
import os
import tensorflow as tf
import tensorflow_ranking as tfr
from official.nlp.configs import encoders
from tensorflow_ranking.extension.premade import tfrbert_task
2026-10-31 11:23:38.878153: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2026-10-31 11:23:38.878210: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2026-10-31 11:23:38.879746: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/google/api_core/_python_version_support.py:273: FutureWarning: You are using a Python version (3.11.14) which Google will stop supporting in new releases of google.api_core once it reaches its end of life (2027-10-24). Please upgrade to the latest Python version, or at least Python 3.12, to continue receiving updates for google.api_core past that date.
warnings.warn(message, FutureWarning)
2026-10-31 11:23:40.898973: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_hub/__init__.py:61: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
from pkg_resources import parse_version
ERROR:absl:cannot import name 'runtime_version' from 'google.protobuf' (/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/google/protobuf/__init__.py)
Traceback (most recent call last):
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_datasets/__init__.py", line 79, in <module>
from tensorflow_datasets import rlds # pylint: disable=g-bad-import-order
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_datasets/rlds/__init__.py", line 21, in <module>
from tensorflow_datasets.rlds import envlogger_reader
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_datasets/rlds/envlogger_reader.py", line 21, in <module>
from tensorflow_datasets.core.utils.lazy_imports_utils import tree
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_datasets/core/__init__.py", line 21, in <module>
from tensorflow_datasets.core import community
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_datasets/core/community/__init__.py", line 19, in <module>
from tensorflow_datasets.core.community.huggingface_wrapper import mock_builtin_to_use_gfile
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_datasets/core/community/huggingface_wrapper.py", line 32, in <module>
from tensorflow_datasets.core import dataset_builder
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_datasets/core/dataset_builder.py", line 162, in <module>
class DatasetBuilder(registered.RegisteredDataset):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/etils/epy/lazy_imports_utils.py", line 123, in __getattr__
return getattr(self._module, name)
^^^^^^^^^^^^
File "/home/kbuilder/.pyenv-v2.6.18/versions/3.11.14/lib/python3.11/functools.py", line 1001, in __get__
val = self.func(instance)
^^^^^^^^^^^^^^^^^^^
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/etils/epy/lazy_imports_utils.py", line 84, in _module
module = importlib.import_module(self.module_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/kbuilder/.pyenv-v2.6.18/versions/3.11.14/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_datasets/core/registered.py", line 36, in <module>
import tensorflow_datasets.core.logging as _tfds_logging
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_datasets/core/logging/__init__.py", line 29, in <module>
from tensorflow_datasets.core.logging import logging_logger
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_datasets/core/logging/logging_logger.py", line 21, in <module>
from tensorflow_datasets.core import decode
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_datasets/core/decode/__init__.py", line 22, in <module>
from tensorflow_datasets.core.decode.partial_decode import PartialDecoding
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_datasets/core/decode/partial_decode.py", line 21, in <module>
from tensorflow_datasets.core import features as features_lib
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_datasets/core/features/__init__.py", line 22, in <module>
from tensorflow_datasets.core.features.audio_feature import Audio
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_datasets/core/features/audio_feature.py", line 33, in <module>
from tensorflow_datasets.core.features import feature as feature_lib
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_datasets/core/features/feature.py", line 36, in <module>
from tensorflow_datasets.core.proto import feature_pb2
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_datasets/core/proto/__init__.py", line 18, in <module>
from tensorflow_datasets.core.proto import dataset_info_generated_pb2 as dataset_info_pb2 # pylint: disable=line-too-long
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_datasets/core/proto/dataset_info_generated_pb2.py", line 32, in <module>
from tensorflow_metadata.proto.v0 import schema_pb2 as tensorflow__metadata_dot_proto_dot_v0_dot_schema__pb2
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_metadata/proto/__init__.py", line 16, in <module>
from tensorflow_metadata.proto.v0 import (
File "/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/tensorflow_metadata/proto/v0/anomalies_pb2.py", line 9, in <module>
from google.protobuf import runtime_version as _runtime_version
ImportError: cannot import name 'runtime_version' from 'google.protobuf' (/tmpfs/src/tf_docs_env/lib/python3.11/site-packages/google/protobuf/__init__.py)
Data preparation
Download training and test data.
wget -O "/tmp/train.tfrecords" "https://ciir.cs.umass.edu/downloads/Antique/tf-ranking/antique_train_seq_64_elwc.tfrecords"wget -O "/tmp/test.tfrecords" "https://ciir.cs.umass.edu/downloads/Antique/tf-ranking/antique_test_seq_64_elwc.tfrecords"
--2026-10-31 11:23:42-- https://ciir.cs.umass.edu/downloads/Antique/tf-ranking/antique_train_seq_64_elwc.tfrecords Resolving ciir.cs.umass.edu (ciir.cs.umass.edu)... 128.119.246.154 Connecting to ciir.cs.umass.edu (ciir.cs.umass.edu)|128.119.246.154|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 8743528 (8.3M) Saving to: ‘/tmp/train.tfrecords’ /tmp/train.tfrecord 0%[ ] 0 --.-KB/s /tmp/train.tfrecord 13%[=> ] 1.14M 5.68MB/s /tmp/train.tfrecord 100%[===================>] 8.34M 26.2MB/s in 0.3s 2026-10-31 11:23:43 (26.2 MB/s) - ‘/tmp/train.tfrecords’ saved [8743528/8743528] --2026-10-31 11:23:43-- https://ciir.cs.umass.edu/downloads/Antique/tf-ranking/antique_test_seq_64_elwc.tfrecords Resolving ciir.cs.umass.edu (ciir.cs.umass.edu)... 128.119.246.154 Connecting to ciir.cs.umass.edu (ciir.cs.umass.edu)|128.119.246.154|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 692072 (676K) Saving to: ‘/tmp/test.tfrecords’ /tmp/test.tfrecords 0%[ ] 0 --.-KB/s /tmp/test.tfrecords 100%[===================>] 675.85K 4.03MB/s in 0.2s 2026-10-31 11:23:43 (4.03 MB/s) - ‘/tmp/test.tfrecords’ saved [692072/692072]
mkdir -p /tmp/tfrbertwget "https://storage.googleapis.com/cloud-tpu-checkpoints/bert/v3/uncased_L-12_H-768_A-12.tar.gz" -P "/tmp/tfrbert"mkdir -p /tmp/tfrbert/uncased_L-12_H-768_A-12tar -xvf /tmp/tfrbert/uncased_L-12_H-768_A-12.tar.gz --strip-components 3 -C "/tmp/tfrbert/uncased_L-12_H-768_A-12/"
--2026-10-31 11:23:43-- https://storage.googleapis.com/cloud-tpu-checkpoints/bert/v3/uncased_L-12_H-768_A-12.tar.gz
Resolving storage.googleapis.com (storage.googleapis.com)... 192.178.239.207, 142.251.107.207, 74.125.26.207, ...
Connecting to storage.googleapis.com (storage.googleapis.com)|192.178.239.207|:443... connected.
HTTP request sent, awaiting response...
200 OK
Length: 405351189 (387M) [application/octet-stream]
Saving to: ‘/tmp/tfrbert/uncased_L-12_H-768_A-12.tar.gz’
uncased_L 0%[ ] 0 --.-KB/s
uncased_L- 2%[ ] 8.01M 39.9MB/s
uncased_L-1 7%[> ] 29.52M 73.6MB/s
uncased_L-12 12%[=> ] 47.48M 79.0MB/s
uncased_L-12_ 14%[=> ] 56.01M 67.1MB/s
uncased_L-12_H 20%[===> ] 78.15M 75.5MB/s
uncased_L-12_H- 24%[===> ] 94.97M 76.9MB/s
uncased_L-12_H-7 28%[====> ] 112.07M 78.1MB/s
uncased_L-12_H-76 34%[=====> ] 134.42M 82.2MB/s
uncased_L-12_H-768 39%[======> ] 152.01M 80.8MB/s
uncased_L-12_H-768_ 42%[=======> ] 164.04M 78.8MB/s
ncased_L-12_H-768_A 45%[========> ] 176.95M 77.6MB/s
cased_L-12_H-768_A- 50%[=========> ] 195.28M 78.7MB/s
ased_L-12_H-768_A-1 57%[==========> ] 222.02M 82.8MB/s
sed_L-12_H-768_A-12 65%[============> ] 253.52M 87.9MB/s
ed_L-12_H-768_A-12. 69%[============> ] 270.55M 87.7MB/s eta 1s
d_L-12_H-768_A-12.t 71%[=============> ] 275.66M 83.9MB/s eta 1s
_L-12_H-768_A-12.ta 76%[==============> ] 297.44M 86.9MB/s eta 1s
L-12_H-768_A-12.tar 82%[===============> ] 318.67M 88.4MB/s eta 1s
-12_H-768_A-12.tar. 90%[=================> ] 349.21M 90.8MB/s eta 1s
12_H-768_A-12.tar.g 97%[==================> ] 376.12M 98.1MB/s eta 0s
uncased_L-12_H-768_ 100%[===================>] 386.57M 99.3MB/s in 4.2s
2026-10-31 11:23:48 (93.1 MB/s) - ‘/tmp/tfrbert/uncased_L-12_H-768_A-12.tar.gz’ saved [405351189/405351189]
tmp/temp_dir/raw/vocab.txt
tmp/temp_dir/raw/bert_model.ckpt.index
tmp/temp_dir/raw/bert_model.ckpt.data-00000-of-00001
tmp/temp_dir/raw/bert_config.json
Overview of TFR-BERT in Orbit
BERT-based ranking models (TFR-BERT) have been shown to be effective for learning-to-rank tasks when using raw textual features for query and passages in MSMARCO passage ranking dataset.
Orbit is a flexible, lightweight library designed to make it easy to write custom training loops in TensorFlow. TensorFlow Ranking provides support for implementing ranking models, particularly for BERT based ranking models using Orbit.
Create a Ranking Task for TFR-BERT
We create a ranking task for TFR-BERT model which can be trained using Orbit. The steps to build this are:
- Define Feature Specifications
- Define datasets
- Setup data and task configurations
Specify Features
Feature Specification are TensorFlow abstractions to capture information about each feature. These help developers and model researchers understand and use a model.
Create feature specifications for context features, example features, and labels, consistent with the input formats for ranking, such as ELWC format.
SEQ_LENGTH = 64
context_feature_spec = {}
example_feature_spec = {
'input_word_ids': tf.io.FixedLenFeature(
shape=(SEQ_LENGTH,), dtype=tf.int64,
default_value=[0] * SEQ_LENGTH),
'input_mask': tf.io.FixedLenFeature(
shape=(SEQ_LENGTH,), dtype=tf.int64,
default_value=[0] * SEQ_LENGTH),
'input_type_ids': tf.io.FixedLenFeature(
shape=(SEQ_LENGTH,), dtype=tf.int64,
default_value=[0] * SEQ_LENGTH)}
label_spec = (
"relevance",
tf.io.FixedLenFeature(shape=(1,), dtype=tf.int64, default_value=-1)
)
Define Datasets
We define data configurations for training and validation data, which specifies parameters such as path, batch size, and dataset format. These configurations are used to create training and validation datasets.
# Set up data config
# We use a small list size here for demo purposes only. Users can use a larger
# list size on a machine with more memory to train TFR-BERT.
train_data_config = tfrbert_task.TFRBertDataConfig(
input_path="/tmp/train.tfrecords",
is_training=True,
global_batch_size=8,
list_size=2,
dataset_fn='tfrecord',
seq_length=64)
validation_data_config = tfrbert_task.TFRBertDataConfig(
input_path="/tmp/test.tfrecords",
is_training=False,
global_batch_size=8,
list_size=2,
dataset_fn='tfrecord',
seq_length=64)
Define Task
We define a task configuration which defines the training and validation dataset along with the model. This configuration creates a TFRBertTask object that can be trained using Orbit.
# Set up task config
task_config = tfrbert_task.TFRBertConfig(
init_checkpoint='/tmp/tfrbert/uncased_L-12_H-768_A-12/bert_model.ckpt',
train_data=train_data_config,
validation_data=validation_data_config,
model=tfrbert_task.TFRBertModelConfig(
encoder=encoders.EncoderConfig(
bert=encoders.BertEncoderConfig(num_layers=12))))
# Set up TFRBertTask
task = tfrbert_task.TFRBertTask(
task_config,
label_spec=label_spec,
dataset_fn=tf.data.TFRecordDataset,
logging_dir='/tmp/model_dir')
Train and evaluate the model
We define the training loop here to train and evaluate the model. We define the metrics, create train and eval datasets and train the model for a specific number of training steps.
metrics = task.build_metrics()
model = task.build_model()
task.initialize(model)
train_dataset = task.build_inputs(task_config.train_data)
vali_dataset = task.build_inputs(task_config.validation_data)
train_iterator = iter(train_dataset)
vali_iterator = iter(vali_dataset)
optimizer = tf.keras.optimizers.Adam(lr=1e-6)
NUM_TRAIN_STEPS = 100
EVAL_STEPS = 10
for train_step in range(NUM_TRAIN_STEPS):
task.train_step(next(train_iterator), model, optimizer, metrics=metrics)
train_metrics = {m.name: m.result().numpy() for m in metrics}
print("Training metrics for epoch: " + str(train_step) + " ", train_metrics)
if train_step % EVAL_STEPS == 0:
task.validation_step(next(train_iterator), model, metrics=metrics)
vali_metrics = {m.name: m.result().numpy() for m in metrics}
print("Validation metrics for epoch: " + str(train_step) + " ",
vali_metrics)
2026-10-31 11:23:53.378873: E external/local_xla/xla/stream_executor/cuda/cuda_driver.cc:274] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
WARNING:absl:`lr` is deprecated in Keras optimizer, please use `learning_rate` or use the legacy optimizer, e.g.,tf.keras.optimizers.legacy.Adam.
Training metrics for epoch: 0 {'MAP': 0.9375, 'NDCG@1': 0.73214287, 'NDCG@5': 0.912364, 'NDCG@10': 0.912364, 'MRR@1': 0.875, 'MRR@5': 0.9375, 'MRR@10': 0.9375}
Validation metrics for epoch: 0 {'MAP': 0.96875, 'NDCG@1': 0.7529762, 'NDCG@5': 0.92272514, 'NDCG@10': 0.92272514, 'MRR@1': 0.9375, 'MRR@5': 0.96875, 'MRR@10': 0.96875}
Training metrics for epoch: 1 {'MAP': 0.9791667, 'NDCG@1': 0.7162698, 'NDCG@5': 0.91046685, 'NDCG@10': 0.91046685, 'MRR@1': 0.9583333, 'MRR@5': 0.9791667, 'MRR@10': 0.9791667}
Training metrics for epoch: 2 {'MAP': 0.984375, 'NDCG@1': 0.70386904, 'NDCG@5': 0.90705323, 'NDCG@10': 0.90705323, 'MRR@1': 0.96875, 'MRR@5': 0.984375, 'MRR@10': 0.984375}
Training metrics for epoch: 3 {'MAP': 0.9875, 'NDCG@1': 0.7464286, 'NDCG@5': 0.92056024, 'NDCG@10': 0.92056024, 'MRR@1': 0.975, 'MRR@5': 0.9875, 'MRR@10': 0.9875}
Training metrics for epoch: 4 {'MAP': 0.9895833, 'NDCG@1': 0.7589286, 'NDCG@5': 0.9242961, 'NDCG@10': 0.9242961, 'MRR@1': 0.9791667, 'MRR@5': 0.9895833, 'MRR@10': 0.9895833}
Training metrics for epoch: 5 {'MAP': 0.9910714, 'NDCG@1': 0.7933674, 'NDCG@5': 0.9351109, 'NDCG@10': 0.9351109, 'MRR@1': 0.98214287, 'MRR@5': 0.9910714, 'MRR@10': 0.9910714}
Training metrics for epoch: 6 {'MAP': 0.9921875, 'NDCG@1': 0.7790179, 'NDCG@5': 0.9309062, 'NDCG@10': 0.9309062, 'MRR@1': 0.984375, 'MRR@5': 0.9921875, 'MRR@10': 0.9921875}
Training metrics for epoch: 7 {'MAP': 0.9861111, 'NDCG@1': 0.76851857, 'NDCG@5': 0.9266033, 'NDCG@10': 0.9266033, 'MRR@1': 0.9722222, 'MRR@5': 0.9861111, 'MRR@10': 0.9861111}
Training metrics for epoch: 8 {'MAP': 0.9875, 'NDCG@1': 0.7702381, 'NDCG@5': 0.9266882, 'NDCG@10': 0.9266882, 'MRR@1': 0.975, 'MRR@5': 0.9875, 'MRR@10': 0.9875}
Training metrics for epoch: 9 {'MAP': 0.98295456, 'NDCG@1': 0.77326846, 'NDCG@5': 0.92727244, 'NDCG@10': 0.92727244, 'MRR@1': 0.96590906, 'MRR@5': 0.98295456, 'MRR@10': 0.98295456}
Training metrics for epoch: 10 {'MAP': 0.9791667, 'NDCG@1': 0.75793654, 'NDCG@5': 0.92171365, 'NDCG@10': 0.92171365, 'MRR@1': 0.9583333, 'MRR@5': 0.9791667, 'MRR@10': 0.9791667}
Validation metrics for epoch: 10 {'MAP': 0.97596157, 'NDCG@1': 0.73946893, 'NDCG@5': 0.91541386, 'NDCG@10': 0.91541386, 'MRR@1': 0.9519231, 'MRR@5': 0.97596157, 'MRR@10': 0.97596157}
Training metrics for epoch: 11 {'MAP': 0.9776786, 'NDCG@1': 0.7478742, 'NDCG@5': 0.9184913, 'NDCG@10': 0.9184913, 'MRR@1': 0.95535713, 'MRR@5': 0.9776786, 'MRR@10': 0.9776786}
Training metrics for epoch: 12 {'MAP': 0.975, 'NDCG@1': 0.7444445, 'NDCG@5': 0.9170479, 'NDCG@10': 0.9170479, 'MRR@1': 0.95, 'MRR@5': 0.975, 'MRR@10': 0.975}
Training metrics for epoch: 13 {'MAP': 0.9765625, 'NDCG@1': 0.7552084, 'NDCG@5': 0.9206442, 'NDCG@10': 0.9206442, 'MRR@1': 0.953125, 'MRR@5': 0.9765625, 'MRR@10': 0.9765625}
Training metrics for epoch: 14 {'MAP': 0.97794116, 'NDCG@1': 0.75070035, 'NDCG@5': 0.9195165, 'NDCG@10': 0.9195165, 'MRR@1': 0.9558824, 'MRR@5': 0.97794116, 'MRR@10': 0.97794116}
Training metrics for epoch: 15 {'MAP': 0.9756944, 'NDCG@1': 0.7536376, 'NDCG@5': 0.920272, 'NDCG@10': 0.920272, 'MRR@1': 0.9513889, 'MRR@5': 0.9756944, 'MRR@10': 0.9756944}
Training metrics for epoch: 16 {'MAP': 0.9769737, 'NDCG@1': 0.74216795, 'NDCG@5': 0.9170982, 'NDCG@10': 0.9170982, 'MRR@1': 0.95394737, 'MRR@5': 0.9769737, 'MRR@10': 0.9769737}
Training metrics for epoch: 17 {'MAP': 0.978125, 'NDCG@1': 0.74196434, 'NDCG@5': 0.9171215, 'NDCG@10': 0.9171215, 'MRR@1': 0.95625, 'MRR@5': 0.978125, 'MRR@10': 0.978125}
Training metrics for epoch: 18 {'MAP': 0.9791667, 'NDCG@1': 0.7508504, 'NDCG@5': 0.9200799, 'NDCG@10': 0.9200799, 'MRR@1': 0.9583333, 'MRR@5': 0.9791667, 'MRR@10': 0.9791667}
Training metrics for epoch: 19 {'MAP': 0.9801136, 'NDCG@1': 0.7621754, 'NDCG@5': 0.9237127, 'NDCG@10': 0.9237127, 'MRR@1': 0.96022725, 'MRR@5': 0.9801136, 'MRR@10': 0.9801136}
Training metrics for epoch: 20 {'MAP': 0.98097825, 'NDCG@1': 0.76630443, 'NDCG@5': 0.9252251, 'NDCG@10': 0.9252251, 'MRR@1': 0.9619565, 'MRR@5': 0.98097825, 'MRR@10': 0.98097825}
Validation metrics for epoch: 20 {'MAP': 0.9817708, 'NDCG@1': 0.76215285, 'NDCG@5': 0.9240412, 'NDCG@10': 0.9240412, 'MRR@1': 0.9635417, 'MRR@5': 0.9817708, 'MRR@10': 0.9817708}
Training metrics for epoch: 21 {'MAP': 0.98, 'NDCG@1': 0.75952387, 'NDCG@5': 0.9229532, 'NDCG@10': 0.9229532, 'MRR@1': 0.96, 'MRR@5': 0.98, 'MRR@10': 0.98}
Training metrics for epoch: 22 {'MAP': 0.9807692, 'NDCG@1': 0.76602566, 'NDCG@5': 0.9251184, 'NDCG@10': 0.9251184, 'MRR@1': 0.96153843, 'MRR@5': 0.9807692, 'MRR@10': 0.9807692}
Training metrics for epoch: 23 {'MAP': 0.9791667, 'NDCG@1': 0.7647708, 'NDCG@5': 0.924646, 'NDCG@10': 0.924646, 'MRR@1': 0.9583333, 'MRR@5': 0.9791667, 'MRR@10': 0.9791667}
Training metrics for epoch: 24 {'MAP': 0.97991073, 'NDCG@1': 0.7693453, 'NDCG@5': 0.9260417, 'NDCG@10': 0.9260417, 'MRR@1': 0.9598214, 'MRR@5': 0.97991073, 'MRR@10': 0.97991073}
Training metrics for epoch: 25 {'MAP': 0.98060346, 'NDCG@1': 0.7748358, 'NDCG@5': 0.9278764, 'NDCG@10': 0.9278764, 'MRR@1': 0.9612069, 'MRR@5': 0.98060346, 'MRR@10': 0.98060346}
Training metrics for epoch: 26 {'MAP': 0.9791667, 'NDCG@1': 0.7698413, 'NDCG@5': 0.92615026, 'NDCG@10': 0.92615026, 'MRR@1': 0.9583333, 'MRR@5': 0.9791667, 'MRR@10': 0.9791667}
Training metrics for epoch: 27 {'MAP': 0.9798387, 'NDCG@1': 0.7611368, 'NDCG@5': 0.9235147, 'NDCG@10': 0.9235147, 'MRR@1': 0.9596774, 'MRR@5': 0.9798387, 'MRR@10': 0.9798387}
Training metrics for epoch: 28 {'MAP': 0.98046875, 'NDCG@1': 0.76190484, 'NDCG@5': 0.92395943, 'NDCG@10': 0.92395943, 'MRR@1': 0.9609375, 'MRR@5': 0.98046875, 'MRR@10': 0.98046875}
Training metrics for epoch: 29 {'MAP': 0.9791667, 'NDCG@1': 0.75775623, 'NDCG@5': 0.9225089, 'NDCG@10': 0.9225089, 'MRR@1': 0.9583333, 'MRR@5': 0.9791667, 'MRR@10': 0.9791667}
Training metrics for epoch: 30 {'MAP': 0.9797794, 'NDCG@1': 0.7596289, 'NDCG@5': 0.92311084, 'NDCG@10': 0.92311084, 'MRR@1': 0.95955884, 'MRR@5': 0.9797794, 'MRR@10': 0.9797794}
Validation metrics for epoch: 30 {'MAP': 0.98035717, 'NDCG@1': 0.7600341, 'NDCG@5': 0.92398876, 'NDCG@10': 0.92398876, 'MRR@1': 0.9607143, 'MRR@5': 0.98035717, 'MRR@10': 0.98035717}
Training metrics for epoch: 31 {'MAP': 0.9809028, 'NDCG@1': 0.75876325, 'NDCG@5': 0.9237945, 'NDCG@10': 0.9237945, 'MRR@1': 0.9618056, 'MRR@5': 0.9809028, 'MRR@10': 0.9809028}
Training metrics for epoch: 32 {'MAP': 0.9814189, 'NDCG@1': 0.7601352, 'NDCG@5': 0.924187, 'NDCG@10': 0.924187, 'MRR@1': 0.9628378, 'MRR@5': 0.9814189, 'MRR@10': 0.9814189}
Training metrics for epoch: 33 {'MAP': 0.9802632, 'NDCG@1': 0.76315796, 'NDCG@5': 0.924968, 'NDCG@10': 0.924968, 'MRR@1': 0.9605263, 'MRR@5': 0.9802632, 'MRR@10': 0.9802632}
Training metrics for epoch: 34 {'MAP': 0.9807692, 'NDCG@1': 0.7646521, 'NDCG@5': 0.9254297, 'NDCG@10': 0.9254297, 'MRR@1': 0.96153843, 'MRR@5': 0.9807692, 'MRR@10': 0.9807692}
Training metrics for epoch: 35 {'MAP': 0.98125, 'NDCG@1': 0.76607144, 'NDCG@5': 0.92586833, 'NDCG@10': 0.92586833, 'MRR@1': 0.9625, 'MRR@5': 0.98125, 'MRR@10': 0.98125}
Training metrics for epoch: 36 {'MAP': 0.98170733, 'NDCG@1': 0.76306623, 'NDCG@5': 0.92489475, 'NDCG@10': 0.92489475, 'MRR@1': 0.9634146, 'MRR@5': 0.98170733, 'MRR@10': 0.98170733}
Training metrics for epoch: 37 {'MAP': 0.98214287, 'NDCG@1': 0.76105446, 'NDCG@5': 0.9243371, 'NDCG@10': 0.9243371, 'MRR@1': 0.96428573, 'MRR@5': 0.98214287, 'MRR@10': 0.98214287}
Training metrics for epoch: 38 {'MAP': 0.98255813, 'NDCG@1': 0.76495016, 'NDCG@5': 0.9256141, 'NDCG@10': 0.9256141, 'MRR@1': 0.96511626, 'MRR@5': 0.98255813, 'MRR@10': 0.98255813}
Training metrics for epoch: 39 {'MAP': 0.98153406, 'NDCG@1': 0.7650162, 'NDCG@5': 0.9254318, 'NDCG@10': 0.9254318, 'MRR@1': 0.9630682, 'MRR@5': 0.98153406, 'MRR@10': 0.98153406}
Training metrics for epoch: 40 {'MAP': 0.98194444, 'NDCG@1': 0.76626986, 'NDCG@5': 0.9258216, 'NDCG@10': 0.9258216, 'MRR@1': 0.9638889, 'MRR@5': 0.98194444, 'MRR@10': 0.98194444}
Validation metrics for epoch: 40 {'MAP': 0.9782609, 'NDCG@1': 0.76591617, 'NDCG@5': 0.9229253, 'NDCG@10': 0.92371386, 'MRR@1': 0.9592391, 'MRR@5': 0.9782609, 'MRR@10': 0.9782609}
Training metrics for epoch: 41 {'MAP': 0.9787234, 'NDCG@1': 0.762538, 'NDCG@5': 0.92202735, 'NDCG@10': 0.9227991, 'MRR@1': 0.9601064, 'MRR@5': 0.9787234, 'MRR@10': 0.9787234}
Training metrics for epoch: 42 {'MAP': 0.9791667, 'NDCG@1': 0.76748514, 'NDCG@5': 0.92365175, 'NDCG@10': 0.9244075, 'MRR@1': 0.9609375, 'MRR@5': 0.9791667, 'MRR@10': 0.9791667}
Training metrics for epoch: 43 {'MAP': 0.97959185, 'NDCG@1': 0.7693149, 'NDCG@5': 0.9243629, 'NDCG@10': 0.9251032, 'MRR@1': 0.9617347, 'MRR@5': 0.97959185, 'MRR@10': 0.97959185}
Training metrics for epoch: 44 {'MAP': 0.98, 'NDCG@1': 0.7710715, 'NDCG@5': 0.92504555, 'NDCG@10': 0.925771, 'MRR@1': 0.9625, 'MRR@5': 0.98, 'MRR@10': 0.98}
Training metrics for epoch: 45 {'MAP': 0.98039216, 'NDCG@1': 0.7706583, 'NDCG@5': 0.9249902, 'NDCG@10': 0.9257015, 'MRR@1': 0.9632353, 'MRR@5': 0.98039216, 'MRR@10': 0.98039216}
Training metrics for epoch: 46 {'MAP': 0.9795673, 'NDCG@1': 0.76854396, 'NDCG@5': 0.92415035, 'NDCG@10': 0.92484796, 'MRR@1': 0.96153843, 'MRR@5': 0.9795673, 'MRR@10': 0.9795673}
Training metrics for epoch: 47 {'MAP': 0.9799528, 'NDCG@1': 0.7661725, 'NDCG@5': 0.9234296, 'NDCG@10': 0.9241141, 'MRR@1': 0.9622642, 'MRR@5': 0.9799528, 'MRR@10': 0.9799528}
Training metrics for epoch: 48 {'MAP': 0.9803241, 'NDCG@1': 0.765873, 'NDCG@5': 0.92340726, 'NDCG@10': 0.92407906, 'MRR@1': 0.962963, 'MRR@5': 0.9803241, 'MRR@10': 0.9803241}
Training metrics for epoch: 49 {'MAP': 0.98068184, 'NDCG@1': 0.76688313, 'NDCG@5': 0.923763, 'NDCG@10': 0.9244226, 'MRR@1': 0.96363634, 'MRR@5': 0.98068184, 'MRR@10': 0.98068184}
Training metrics for epoch: 50 {'MAP': 0.98102677, 'NDCG@1': 0.76573133, 'NDCG@5': 0.9233751, 'NDCG@10': 0.924023, 'MRR@1': 0.96428573, 'MRR@5': 0.98102677, 'MRR@10': 0.98102677}
Validation metrics for epoch: 50 {'MAP': 0.98135966, 'NDCG@1': 0.76461995, 'NDCG@5': 0.92245334, 'NDCG@10': 0.9230898, 'MRR@1': 0.9649123, 'MRR@5': 0.98135966, 'MRR@10': 0.98135966}
Training metrics for epoch: 51 {'MAP': 0.98168105, 'NDCG@1': 0.76252055, 'NDCG@5': 0.92182404, 'NDCG@10': 0.9224495, 'MRR@1': 0.9655172, 'MRR@5': 0.98168105, 'MRR@10': 0.98168105}
Training metrics for epoch: 52 {'MAP': 0.9819915, 'NDCG@1': 0.76109767, 'NDCG@5': 0.9214791, 'NDCG@10': 0.922094, 'MRR@1': 0.9661017, 'MRR@5': 0.9819915, 'MRR@10': 0.9819915}
Training metrics for epoch: 53 {'MAP': 0.98229164, 'NDCG@1': 0.7609127, 'NDCG@5': 0.9214915, 'NDCG@10': 0.92209613, 'MRR@1': 0.96666664, 'MRR@5': 0.98229164, 'MRR@10': 0.98229164}
Training metrics for epoch: 54 {'MAP': 0.982582, 'NDCG@1': 0.76014835, 'NDCG@5': 0.9214178, 'NDCG@10': 0.9220125, 'MRR@1': 0.9672131, 'MRR@5': 0.982582, 'MRR@10': 0.982582}
Training metrics for epoch: 55 {'MAP': 0.98185486, 'NDCG@1': 0.7573925, 'NDCG@5': 0.9204364, 'NDCG@10': 0.9210215, 'MRR@1': 0.9657258, 'MRR@5': 0.98185486, 'MRR@10': 0.98185486}
Training metrics for epoch: 56 {'MAP': 0.98214287, 'NDCG@1': 0.75840896, 'NDCG@5': 0.9207941, 'NDCG@10': 0.92136997, 'MRR@1': 0.96626985, 'MRR@5': 0.98214287, 'MRR@10': 0.98214287}
Training metrics for epoch: 57 {'MAP': 0.9824219, 'NDCG@1': 0.7593937, 'NDCG@5': 0.9211407, 'NDCG@10': 0.9217075, 'MRR@1': 0.9667969, 'MRR@5': 0.9824219, 'MRR@10': 0.9824219}
Training metrics for epoch: 58 {'MAP': 0.9826923, 'NDCG@1': 0.7619964, 'NDCG@5': 0.9220346, 'NDCG@10': 0.92259276, 'MRR@1': 0.9673077, 'MRR@5': 0.9826923, 'MRR@10': 0.9826923}
Training metrics for epoch: 59 {'MAP': 0.9810606, 'NDCG@1': 0.759109, 'NDCG@5': 0.9209539, 'NDCG@10': 0.9215036, 'MRR@1': 0.9640151, 'MRR@5': 0.9810606, 'MRR@10': 0.9810606}
Training metrics for epoch: 60 {'MAP': 0.98134327, 'NDCG@1': 0.75897306, 'NDCG@5': 0.9209728, 'NDCG@10': 0.92151433, 'MRR@1': 0.9645522, 'MRR@5': 0.98134327, 'MRR@10': 0.98134327}
Validation metrics for epoch: 60 {'MAP': 0.9816176, 'NDCG@1': 0.76041675, 'NDCG@5': 0.92152464, 'NDCG@10': 0.9213793, 'MRR@1': 0.9650735, 'MRR@5': 0.9816176, 'MRR@10': 0.9816176}
Training metrics for epoch: 61 {'MAP': 0.98188406, 'NDCG@1': 0.76130104, 'NDCG@5': 0.92183554, 'NDCG@10': 0.92169225, 'MRR@1': 0.9655797, 'MRR@5': 0.98188406, 'MRR@10': 0.98188406}
Training metrics for epoch: 62 {'MAP': 0.98214287, 'NDCG@1': 0.76062936, 'NDCG@5': 0.9216193, 'NDCG@10': 0.92147815, 'MRR@1': 0.9660714, 'MRR@5': 0.98214287, 'MRR@10': 0.98214287}
Training metrics for epoch: 63 {'MAP': 0.98239434, 'NDCG@1': 0.76249176, 'NDCG@5': 0.9222124, 'NDCG@10': 0.92207325, 'MRR@1': 0.9665493, 'MRR@5': 0.98239434, 'MRR@10': 0.98239434}
Training metrics for epoch: 64 {'MAP': 0.9817708, 'NDCG@1': 0.76207024, 'NDCG@5': 0.92207557, 'NDCG@10': 0.92193836, 'MRR@1': 0.9652778, 'MRR@5': 0.9817708, 'MRR@10': 0.9817708}
Training metrics for epoch: 65 {'MAP': 0.98202056, 'NDCG@1': 0.76386184, 'NDCG@5': 0.9226462, 'NDCG@10': 0.92251086, 'MRR@1': 0.96575344, 'MRR@5': 0.98202056, 'MRR@10': 0.98202056}
Training metrics for epoch: 66 {'MAP': 0.9814189, 'NDCG@1': 0.7629506, 'NDCG@5': 0.9222975, 'NDCG@10': 0.92216396, 'MRR@1': 0.964527, 'MRR@5': 0.9814189, 'MRR@10': 0.9814189}
Training metrics for epoch: 67 {'MAP': 0.9816667, 'NDCG@1': 0.7632541, 'NDCG@5': 0.9223662, 'NDCG@10': 0.9222345, 'MRR@1': 0.965, 'MRR@5': 0.9816667, 'MRR@10': 0.9816667}
Training metrics for epoch: 68 {'MAP': 0.9819079, 'NDCG@1': 0.76401955, 'NDCG@5': 0.92263746, 'NDCG@10': 0.92250746, 'MRR@1': 0.96546054, 'MRR@5': 0.9819079, 'MRR@10': 0.9819079}
Training metrics for epoch: 69 {'MAP': 0.98214287, 'NDCG@1': 0.7643013, 'NDCG@5': 0.9227, 'NDCG@10': 0.92257166, 'MRR@1': 0.96590906, 'MRR@5': 0.98214287, 'MRR@10': 0.98214287}
Training metrics for epoch: 70 {'MAP': 0.9823718, 'NDCG@1': 0.7662547, 'NDCG@5': 0.9233652, 'NDCG@10': 0.9232385, 'MRR@1': 0.96634614, 'MRR@5': 0.9823718, 'MRR@10': 0.9823718}
Validation metrics for epoch: 70 {'MAP': 0.9825949, 'NDCG@1': 0.766501, 'NDCG@5': 0.92315423, 'NDCG@10': 0.9227665, 'MRR@1': 0.96677214, 'MRR@5': 0.9825949, 'MRR@10': 0.9825949}
Training metrics for epoch: 71 {'MAP': 0.9828125, 'NDCG@1': 0.76674116, 'NDCG@5': 0.92320794, 'NDCG@10': 0.92282504, 'MRR@1': 0.9671875, 'MRR@5': 0.9828125, 'MRR@10': 0.9828125}
Training metrics for epoch: 72 {'MAP': 0.9830247, 'NDCG@1': 0.76829815, 'NDCG@5': 0.9237082, 'NDCG@10': 0.92333007, 'MRR@1': 0.9675926, 'MRR@5': 0.9830247, 'MRR@10': 0.9830247}
Training metrics for epoch: 73 {'MAP': 0.9824695, 'NDCG@1': 0.7650988, 'NDCG@5': 0.92262834, 'NDCG@10': 0.9222548, 'MRR@1': 0.9664634, 'MRR@5': 0.9824695, 'MRR@10': 0.9824695}
Training metrics for epoch: 74 {'MAP': 0.98268074, 'NDCG@1': 0.76792896, 'NDCG@5': 0.92356056, 'NDCG@10': 0.9231915, 'MRR@1': 0.96686745, 'MRR@5': 0.98268074, 'MRR@10': 0.98268074}
Training metrics for epoch: 75 {'MAP': 0.98214287, 'NDCG@1': 0.766936, 'NDCG@5': 0.92318696, 'NDCG@10': 0.9228223, 'MRR@1': 0.9657738, 'MRR@5': 0.98214287, 'MRR@10': 0.98214287}
Training metrics for epoch: 76 {'MAP': 0.9823529, 'NDCG@1': 0.7679972, 'NDCG@5': 0.9236024, 'NDCG@10': 0.92324203, 'MRR@1': 0.96617645, 'MRR@5': 0.9823529, 'MRR@10': 0.9823529}
Training metrics for epoch: 77 {'MAP': 0.98255813, 'NDCG@1': 0.76861846, 'NDCG@5': 0.9238277, 'NDCG@10': 0.9234715, 'MRR@1': 0.9665698, 'MRR@5': 0.98255813, 'MRR@10': 0.98255813}
Training metrics for epoch: 78 {'MAP': 0.98275864, 'NDCG@1': 0.7692255, 'NDCG@5': 0.9240478, 'NDCG@10': 0.9236957, 'MRR@1': 0.96695405, 'MRR@5': 0.98275864, 'MRR@10': 0.98275864}
Training metrics for epoch: 79 {'MAP': 0.98295456, 'NDCG@1': 0.7710362, 'NDCG@5': 0.92467505, 'NDCG@10': 0.92432696, 'MRR@1': 0.96732956, 'MRR@5': 0.98295456, 'MRR@10': 0.98295456}
Training metrics for epoch: 80 {'MAP': 0.9831461, 'NDCG@1': 0.7716024, 'NDCG@5': 0.92488074, 'NDCG@10': 0.9245365, 'MRR@1': 0.9676966, 'MRR@5': 0.9831461, 'MRR@10': 0.9831461}
Validation metrics for epoch: 80 {'MAP': 0.98333335, 'NDCG@1': 0.77294976, 'NDCG@5': 0.9253124, 'NDCG@10': 0.92456895, 'MRR@1': 0.96805555, 'MRR@5': 0.98333335, 'MRR@10': 0.98333335}
Training metrics for epoch: 81 {'MAP': 0.98351645, 'NDCG@1': 0.77348244, 'NDCG@5': 0.92550653, 'NDCG@10': 0.92477125, 'MRR@1': 0.9684066, 'MRR@5': 0.98351645, 'MRR@10': 0.98351645}
Training metrics for epoch: 82 {'MAP': 0.9836956, 'NDCG@1': 0.77270955, 'NDCG@5': 0.92536336, 'NDCG@10': 0.92463607, 'MRR@1': 0.96875, 'MRR@5': 0.9836956, 'MRR@10': 0.9836956}
Training metrics for epoch: 83 {'MAP': 0.98319894, 'NDCG@1': 0.7722733, 'NDCG@5': 0.9252236, 'NDCG@10': 0.9245041, 'MRR@1': 0.9677419, 'MRR@5': 0.98319894, 'MRR@10': 0.98319894}
Training metrics for epoch: 84 {'MAP': 0.98337764, 'NDCG@1': 0.7739361, 'NDCG@5': 0.9257983, 'NDCG@10': 0.9250865, 'MRR@1': 0.9680851, 'MRR@5': 0.98337764, 'MRR@10': 0.98337764}
Training metrics for epoch: 85 {'MAP': 0.98355263, 'NDCG@1': 0.77456135, 'NDCG@5': 0.9260444, 'NDCG@10': 0.9253401, 'MRR@1': 0.96842104, 'MRR@5': 0.98355263, 'MRR@10': 0.98355263}
Training metrics for epoch: 86 {'MAP': 0.98372394, 'NDCG@1': 0.7752976, 'NDCG@5': 0.9263339, 'NDCG@10': 0.92563695, 'MRR@1': 0.96875, 'MRR@5': 0.98372394, 'MRR@10': 0.98372394}
Training metrics for epoch: 87 {'MAP': 0.9838917, 'NDCG@1': 0.7765095, 'NDCG@5': 0.9267194, 'NDCG@10': 0.9260296, 'MRR@1': 0.96907216, 'MRR@5': 0.9838917, 'MRR@10': 0.9838917}
Training metrics for epoch: 88 {'MAP': 0.98341835, 'NDCG@1': 0.77678573, 'NDCG@5': 0.9267847, 'NDCG@10': 0.9261019, 'MRR@1': 0.96811223, 'MRR@5': 0.98341835, 'MRR@10': 0.98341835}
Training metrics for epoch: 89 {'MAP': 0.98358583, 'NDCG@1': 0.7768759, 'NDCG@5': 0.9267914, 'NDCG@10': 0.9261156, 'MRR@1': 0.96843433, 'MRR@5': 0.98358583, 'MRR@10': 0.98358583}
Training metrics for epoch: 90 {'MAP': 0.983125, 'NDCG@1': 0.7771429, 'NDCG@5': 0.9268546, 'NDCG@10': 0.92618555, 'MRR@1': 0.9675, 'MRR@5': 0.983125, 'MRR@10': 0.983125}
Validation metrics for epoch: 90 {'MAP': 0.9832921, 'NDCG@1': 0.7767563, 'NDCG@5': 0.92676264, 'NDCG@10': 0.925741, 'MRR@1': 0.9678218, 'MRR@5': 0.9832921, 'MRR@10': 0.9832921}
Training metrics for epoch: 91 {'MAP': 0.9834559, 'NDCG@1': 0.7781279, 'NDCG@5': 0.9272315, 'NDCG@10': 0.9262199, 'MRR@1': 0.96813726, 'MRR@5': 0.9834559, 'MRR@10': 0.9834559}
Training metrics for epoch: 92 {'MAP': 0.9830097, 'NDCG@1': 0.77837497, 'NDCG@5': 0.9272886, 'NDCG@10': 0.9262868, 'MRR@1': 0.967233, 'MRR@5': 0.9830097, 'MRR@10': 0.9830097}
Training metrics for epoch: 93 {'MAP': 0.98257214, 'NDCG@1': 0.77781594, 'NDCG@5': 0.9271003, 'NDCG@10': 0.9261081, 'MRR@1': 0.96634614, 'MRR@5': 0.98257214, 'MRR@10': 0.98257214}
Training metrics for epoch: 94 {'MAP': 0.9827381, 'NDCG@1': 0.7774377, 'NDCG@5': 0.9270095, 'NDCG@10': 0.92602676, 'MRR@1': 0.96666664, 'MRR@5': 0.9827381, 'MRR@10': 0.9827381}
Training metrics for epoch: 95 {'MAP': 0.9829009, 'NDCG@1': 0.77751577, 'NDCG@5': 0.9271108, 'NDCG@10': 0.9261373, 'MRR@1': 0.9669811, 'MRR@5': 0.9829009, 'MRR@10': 0.9829009}
Training metrics for epoch: 96 {'MAP': 0.98247665, 'NDCG@1': 0.7777593, 'NDCG@5': 0.9271669, 'NDCG@10': 0.92620254, 'MRR@1': 0.9661215, 'MRR@5': 0.98247665, 'MRR@10': 0.98247665}
Training metrics for epoch: 97 {'MAP': 0.9820602, 'NDCG@1': 0.7770062, 'NDCG@5': 0.9268861, 'NDCG@10': 0.9259307, 'MRR@1': 0.9652778, 'MRR@5': 0.9820602, 'MRR@10': 0.9820602}
Training metrics for epoch: 98 {'MAP': 0.98165137, 'NDCG@1': 0.77724993, 'NDCG@5': 0.92694324, 'NDCG@10': 0.92599654, 'MRR@1': 0.9644495, 'MRR@5': 0.98165137, 'MRR@10': 0.98165137}
Training metrics for epoch: 99 {'MAP': 0.9818182, 'NDCG@1': 0.77667755, 'NDCG@5': 0.9267592, 'NDCG@10': 0.9258211, 'MRR@1': 0.9647727, 'MRR@5': 0.9818182, 'MRR@10': 0.9818182}