# clinc_oos

<br />

- **Description**:

Task-oriented dialog systems need to know when a query falls outside their range
of supported intents, but current text classification corpora only define label
sets that cover every example. We introduce a new dataset that includes queries
that are out-of-scope (OOS), i.e., queries that do not fall into any of the
system's supported intents. This poses a new challenge because models cannot
assume that every query at inference time belongs to a system-supported intent
class. Our dataset also covers 150 intent classes over 10 domains, capturing the
breadth that a production task-oriented agent must handle. It offers a way of
more rigorously and realistically benchmarking text classification in
task-driven dialog systems.

- **Additional Documentation** :
  [Explore on Papers With Code](https://paperswithcode.com/dataset/clinc150)

- **Homepage** :
  <https://github.com/clinc/oos-eval/>

- **Source code** :
  [`tfds.text.ClincOOS`](https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/text/clinc_oos.py)

- **Versions**:

  - **`0.1.0`** (default): No release notes.
- **Download size** : `256.01 KiB`

- **Dataset size** : `3.40 MiB`

- **Auto-cached**
  ([documentation](https://www.tensorflow.org/datasets/performances#auto-caching)):
  Yes

- **Splits**:

| Split | Examples |
|---|---|
| `'test'` | 4,500 |
| `'test_oos'` | 1,000 |
| `'train'` | 15,000 |
| `'train_oos'` | 100 |
| `'validation'` | 3,000 |
| `'validation_oos'` | 100 |

- **Feature structure**:

    FeaturesDict({
        'domain': int32,
        'domain_name': Text(shape=(), dtype=string),
        'intent': int32,
        'intent_name': Text(shape=(), dtype=string),
        'text': Text(shape=(), dtype=string),
    })

- **Feature documentation**:

| Feature | Class | Shape | Dtype | Description |
|---|---|---|---|---|
|   | FeaturesDict |   |   |   |
| domain | Tensor |   | int32 |   |
| domain_name | Text |   | string |   |
| intent | Tensor |   | int32 |   |
| intent_name | Text |   | string |   |
| text | Text |   | string |   |

- **Supervised keys** (See
  [`as_supervised` doc](https://www.tensorflow.org/datasets/api_docs/python/tfds/load#args)):
  `('text', 'intent')`

- **Figure**
  ([tfds.show_examples](https://www.tensorflow.org/datasets/api_docs/python/tfds/visualization/show_examples)):
  Not supported.

- **Examples**
  ([tfds.as_dataframe](https://www.tensorflow.org/datasets/api_docs/python/tfds/as_dataframe)):

<button id="displaydataframe">Display examples...</button>

- **Citation**:

    @inproceedings{larson-etal-2019-evaluation,
        title = "An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction",
        author = "Larson, Stefan  and
          Mahendran, Anish  and
          Peper, Joseph J.  and
          Clarke, Christopher  and
          Lee, Andrew  and
          Hill, Parker  and
          Kummerfeld, Jonathan K.  and
          Leach, Kevin  and
          Laurenzano, Michael A.  and
          Tang, Lingjia  and
          Mars, Jason",
        booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)",
        month = nov,
        year = "2019",
        address = "Hong Kong, China",
        publisher = "Association for Computational Linguistics",
        url = "https://www.aclweb.org/anthology/D19-1131",
        doi = "10.18653/v1/D19-1131",
        pages = "1311--1316",
    }