# paws_x_wiki

<br />

- **Description**:

This dataset contains 23,659 human translated PAWS evaluation pairs and 296,406
machine translated training pairs in six typologically distinct languages:

- French
- Spanish
- German
- Chinese
- Japanese
- Korean

For further details, see the accompanying paper: PAWS-X: A Cross-lingual
Adversarial Dataset for Paraphrase Identification at
<https://arxiv.org/abs/1908.11828>

Similar to PAWS Dataset, examples are split into Train/Dev/Test sections. All
files are in the tsv format with four columns:

1. `id`: A unique id for each pair.
2. `sentence1`: The first sentence.
3. `sentence2`: The second sentence.
4. `(noisy_)label`: (Noisy) label for each pair.

Each label has two possible values: 0 indicates the pair has different meaning,
while 1 indicates the pair is a paraphrase.

- **Additional Documentation** :
  [Explore on Papers With Code](https://paperswithcode.com/dataset/paws-x)

- **Homepage** :
  <https://github.com/google-research-datasets/paws/tree/master/pawsx>

- **Source code** :
  [`tfds.datasets.paws_x_wiki.Builder`](https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/datasets/paws_x_wiki/paws_x_wiki_dataset_builder.py)

- **Versions**:

  - **`1.0.0`** (default): No release notes.
- **Download size** : `28.88 MiB`

- **Auto-cached**
  ([documentation](https://www.tensorflow.org/datasets/performances#auto-caching)):
  Yes

- **Feature structure**:

    FeaturesDict({
        'label': ClassLabel(shape=(), dtype=int64, num_classes=2),
        'sentence1': Text(shape=(), dtype=string),
        'sentence2': Text(shape=(), dtype=string),
    })

- **Feature documentation**:

| Feature | Class | Shape | Dtype | Description |
|---|---|---|---|---|
|   | FeaturesDict |   |   |   |
| label | ClassLabel |   | int64 |   |
| sentence1 | Text |   | string |   |
| sentence2 | Text |   | string |   |

- **Supervised keys** (See
  [`as_supervised` doc](https://www.tensorflow.org/datasets/api_docs/python/tfds/load#args)):
  `None`

- **Figure**
  ([tfds.show_examples](https://www.tensorflow.org/datasets/api_docs/python/tfds/visualization/show_examples)):
  Not supported.

- **Citation**:

    @InProceedings{pawsx2019emnlp,
      title = { {PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification} },
      author = {Yang, Yinfei and Zhang, Yuan and Tar, Chris and Baldridge, Jason},
      booktitle = {Proc. of EMNLP},
      year = {2019}
    }

## paws_x_wiki/de (default config)

- **Config description**: Translated to de

- **Dataset size** : `15.27 MiB`

- **Splits**:

| Split | Examples |
|---|---|
| `'test'` | 2,000 |
| `'train'` | 49,380 |
| `'validation'` | 2,000 |

- **Examples** ([tfds.as_dataframe](https://www.tensorflow.org/datasets/api_docs/python/tfds/as_dataframe)):

<button id="displaydataframe">Display examples...</button>

## paws_x_wiki/en

- **Config description**: Translated to en

- **Dataset size** : `14.59 MiB`

- **Splits**:

| Split | Examples |
|---|---|
| `'test'` | 2,000 |
| `'train'` | 49,175 |
| `'validation'` | 2,000 |

- **Examples** ([tfds.as_dataframe](https://www.tensorflow.org/datasets/api_docs/python/tfds/as_dataframe)):

<button id="displaydataframe">Display examples...</button>

## paws_x_wiki/es

- **Config description**: Translated to es

- **Dataset size** : `15.27 MiB`

- **Splits**:

| Split | Examples |
|---|---|
| `'test'` | 2,000 |
| `'train'` | 49,401 |
| `'validation'` | 1,961 |

- **Examples** ([tfds.as_dataframe](https://www.tensorflow.org/datasets/api_docs/python/tfds/as_dataframe)):

<button id="displaydataframe">Display examples...</button>

## paws_x_wiki/fr

- **Config description**: Translated to fr

- **Dataset size** : `15.79 MiB`

- **Splits**:

| Split | Examples |
|---|---|
| `'test'` | 2,000 |
| `'train'` | 49,399 |
| `'validation'` | 1,988 |

- **Examples** ([tfds.as_dataframe](https://www.tensorflow.org/datasets/api_docs/python/tfds/as_dataframe)):

<button id="displaydataframe">Display examples...</button>

## paws_x_wiki/ja

- **Config description**: Translated to ja

- **Dataset size** : `17.77 MiB`

- **Splits**:

| Split | Examples |
|---|---|
| `'test'` | 2,000 |
| `'train'` | 49,401 |
| `'validation'` | 2,000 |

- **Examples** ([tfds.as_dataframe](https://www.tensorflow.org/datasets/api_docs/python/tfds/as_dataframe)):

<button id="displaydataframe">Display examples...</button>

## paws_x_wiki/ko

- **Config description**: Translated to ko

- **Dataset size** : `16.42 MiB`

- **Splits**:

| Split | Examples |
|---|---|
| `'test'` | 1,999 |
| `'train'` | 49,164 |
| `'validation'` | 2,000 |

- **Examples** ([tfds.as_dataframe](https://www.tensorflow.org/datasets/api_docs/python/tfds/as_dataframe)):

<button id="displaydataframe">Display examples...</button>

## paws_x_wiki/zh

- **Config description**: Translated to zh

- **Dataset size** : `13.20 MiB`

- **Splits**:

| Split | Examples |
|---|---|
| `'test'` | 2,000 |
| `'train'` | 49,401 |
| `'validation'` | 2,000 |

- **Examples** ([tfds.as_dataframe](https://www.tensorflow.org/datasets/api_docs/python/tfds/as_dataframe)):

<button id="displaydataframe">Display examples...</button>