# xtreme

<br />

- **Description**:

# Xtreme Benchmark

The Cross-lingual TRansfer Evaluation of Multilingual Encoders (XTREME)
benchmark is a benchmark for the evaluation of the cross-lingual generalization
ability of pre-trained multilingual models. It covers 40 typologically diverse
languages (spanning 12 language families) and includes nine tasks that
collectively require reasoning about different levels of syntax and semantics.
The languages in XTREME are selected to maximize language diversity, coverage in
existing tasks, and availability of training data. Among these are many
under-studied languages, such as the Dravidian languages Tamil (spoken in
southern India, Sri Lanka, and Singapore), Telugu and Malayalam (spoken mainly
in southern India), and the Niger-Congo languages Swahili and Yoruba, spoken in
Africa.

For a full description of the benchmark, see the
[paper](https://arxiv.org/abs/2003.11080).

- **Homepage**:

<https://sites.research.google/xtreme>

- **Versions**:

  - **`1.0.0`** (default): Initial release
- **Datasets in the default version**:

  - `xnli`: [`xtreme_xnli:1.1.0`](https://www.tensorflow.org/datasets/catalog/xtreme_xnli)
  - `pawsx`: [`xtreme_pawsx:1.0.0`](https://www.tensorflow.org/datasets/catalog/xtreme_pawsx)
  - `pos`: [`xtreme_pos:1.0.0`](https://www.tensorflow.org/datasets/catalog/xtreme_pos)
  - `ner`: [`wikiann:1.0.0`](https://www.tensorflow.org/datasets/catalog/wikiann)
  - `xquad`: [`xquad:3.0.0`](https://www.tensorflow.org/datasets/catalog/xquad)
  - `mlqa`: [`mlqa:1.0.0`](https://www.tensorflow.org/datasets/catalog/mlqa)
  - `tydiqa`: [`tydi_qa:3.0.0`](https://www.tensorflow.org/datasets/catalog/tydi_qa)
  - `bucc`: [`bucc:1.0.0`](https://www.tensorflow.org/datasets/catalog/bucc)
  - `tatoeba`: [`tatoeba:1.0.0`](https://www.tensorflow.org/datasets/catalog/tatoeba)
- **Citation**: