- Description:
NEWSROOM is a large dataset for training and evaluating summarization systems. It contains 1.3 million articles and summaries written by authors and editors in the newsrooms of 38 major publications.
Dataset features includes:
- text: Input news text.
- summary: Summary for the news.
And additional features:
- title: news title.
- url: url of the news.
- date: date of the article.
- density: extractive density.
- coverage: extractive coverage.
- compression: compression ratio.
- density_bin: low, medium, high.
- coverage_bin: extractive, abstractive.
- compression_bin: low, medium, high.
This dataset can be downloaded upon requests. Unzip all the contents "train.jsonl, dev.jsonl, test.jsonl" to the tfds folder.
Additional Documentation: Explore on Papers With Code
Homepage: https://summari.es
Source code:
tfds.datasets.newsroom.Builder
Versions:
1.0.0
(default): No release notes.
Download size:
Unknown size
Dataset size:
5.13 GiB
Manual download instructions: This dataset requires you to download the source data manually into
download_config.manual_dir
(defaults to~/tensorflow_datasets/downloads/manual/
):
You should download the dataset from https://summari.es/download/ The webpage requires registration. After downloading, please put dev.jsonl, test.jsonl and train.jsonl files in the manual_dir.Auto-cached (documentation): No
Splits:
Split | Examples |
---|---|
'test' |
108,862 |
'train' |
995,041 |
'validation' |
108,837 |
- Feature structure:
FeaturesDict({
'compression': float32,
'compression_bin': Text(shape=(), dtype=string),
'coverage': float32,
'coverage_bin': Text(shape=(), dtype=string),
'date': Text(shape=(), dtype=string),
'density': float32,
'density_bin': Text(shape=(), dtype=string),
'summary': Text(shape=(), dtype=string),
'text': Text(shape=(), dtype=string),
'title': Text(shape=(), dtype=string),
'url': Text(shape=(), dtype=string),
})
- Feature documentation:
Feature | Class | Shape | Dtype | Description |
---|---|---|---|---|
FeaturesDict | ||||
compression | Tensor | float32 | ||
compression_bin | Text | string | ||
coverage | Tensor | float32 | ||
coverage_bin | Text | string | ||
date | Text | string | ||
density | Tensor | float32 | ||
density_bin | Text | string | ||
summary | Text | string | ||
text | Text | string | ||
title | Text | string | ||
url | Text | string |
Supervised keys (See
as_supervised
doc):('text', 'summary')
Figure (tfds.show_examples): Not supported.
Examples (tfds.as_dataframe):
- Citation:
@article{Grusky_2018,
title={Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies},
url={http://dx.doi.org/10.18653/v1/n18-1065},
DOI={10.18653/v1/n18-1065},
journal={Proceedings of the 2018 Conference of the North American Chapter of
the Association for Computational Linguistics: Human Language
Technologies, Volume 1 (Long Papers)},
publisher={Association for Computational Linguistics},
author={Grusky, Max and Naaman, Mor and Artzi, Yoav},
year={2018}
}