newsroom

• Description:

NEWSROOM is a large dataset for training and evaluating summarization systems. It contains 1.3 million articles and summaries written by authors and editors in the newsrooms of 38 major publications.

Dataset features includes: - text: Input news text. - summary: Summary for the news. And additional features: - title: news title. - url: url of the news. - date: date of the article. - density: extractive density. - coverage: extractive coverage. - compression: compression ratio. - density_bin: low, medium, high. - coverage_bin: extractive, abstractive. - compression_bin: low, medium, high.

This dataset can be downloaded upon requests. Unzip all the contents "train.jsonl, dev.josnl, test.jsonl" to the tfds folder.

• Homepage: https://summari.es

• Source code: tfds.summarization.Newsroom

• Versions:

  • 1.0.0 (default): No release notes.

• Download size: Unknown size

• Dataset size: Unknown size

• Manual download instructions: This dataset requires you to download the source data manually into download_config.manual_dir (defaults to ~/tensorflow_datasets/downloads/manual/):
  You should download the dataset from https://summari.es/download/ The webpage requires registration. After downloading, please put dev.jsonl, test.jsonl and train.jsonl files in the manual_dir.

• Auto-cached (documentation): Unknown

• Splits:

Split	Examples
'test'	108,862
'train'	995,041
'validation'	108,837

• Feature structure:

FeaturesDict({
    'compression': tf.float32,
    'compression_bin': Text(shape=(), dtype=tf.string),
    'coverage': tf.float32,
    'coverage_bin': Text(shape=(), dtype=tf.string),
    'date': Text(shape=(), dtype=tf.string),
    'density': tf.float32,
    'density_bin': Text(shape=(), dtype=tf.string),
    'summary': Text(shape=(), dtype=tf.string),
    'text': Text(shape=(), dtype=tf.string),
    'title': Text(shape=(), dtype=tf.string),
    'url': Text(shape=(), dtype=tf.string),
})

• Feature documentation:

Feature	Class	Shape	Dtype	Description
	FeaturesDict
compression	Tensor		tf.float32
compression_bin	Text		tf.string
coverage	Tensor		tf.float32
coverage_bin	Text		tf.string
date	Text		tf.string
density	Tensor		tf.float32
density_bin	Text		tf.string
summary	Text		tf.string
text	Text		tf.string
title	Text		tf.string
url	Text		tf.string

• Supervised keys (See as_supervised doc): ('text', 'summary')

• Figure (tfds.show_examples): Not supported.

• Examples (tfds.as_dataframe):

• Citation:

@article{Grusky_2018,
   title={Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies},
   url={http://dx.doi.org/10.18653/v1/n18-1065},
   DOI={10.18653/v1/n18-1065},
   journal={Proceedings of the 2018 Conference of the North American Chapter of
          the Association for Computational Linguistics: Human Language
          Technologies, Volume 1 (Long Papers)},
   publisher={Association for Computational Linguistics},
   author={Grusky, Max and Naaman, Mor and Artzi, Yoav},
   year={2018}
}