newsroom

  • Description:

NEWSROOM is a large dataset for training and evaluating summarization systems. It contains 1.3 million articles and summaries written by authors and editors in the newsrooms of 38 major publications.

Dataset features includes:

  • text: Input news text.
  • summary: Summary for the news.

And additional features:

  • title: news title.
  • url: url of the news.
  • date: date of the article.
  • density: extractive density.
  • coverage: extractive coverage.
  • compression: compression ratio.
  • density_bin: low, medium, high.
  • coverage_bin: extractive, abstractive.
  • compression_bin: low, medium, high.

This dataset can be downloaded upon requests. Unzip all the contents "train.jsonl, dev.jsonl, test.jsonl" to the tfds folder.

  • Additional Documentation: Explore on Papers With Code

  • Homepage: https://summari.es

  • Source code: tfds.datasets.newsroom.Builder

  • Versions:

    • 1.0.0 (default): No release notes.
  • Download size: Unknown size

  • Dataset size: 5.13 GiB

  • Manual download instructions: This dataset requires you to download the source data manually into download_config.manual_dir (defaults to ~/tensorflow_datasets/downloads/manual/):
    You should download the dataset from https://summari.es/download/ The webpage requires registration. After downloading, please put dev.jsonl, test.jsonl and train.jsonl files in the manual_dir.

  • Auto-cached (documentation): No

  • Splits:

Split Examples
'test' 108,862
'train' 995,041
'validation' 108,837
  • Feature structure:
FeaturesDict({
    'compression': float32,
    'compression_bin': Text(shape=(), dtype=string),
    'coverage': float32,
    'coverage_bin': Text(shape=(), dtype=string),
    'date': Text(shape=(), dtype=string),
    'density': float32,
    'density_bin': Text(shape=(), dtype=string),
    'summary': Text(shape=(), dtype=string),
    'text': Text(shape=(), dtype=string),
    'title': Text(shape=(), dtype=string),
    'url': Text(shape=(), dtype=string),
})
  • Feature documentation:
Feature Class Shape Dtype Description
FeaturesDict
compression Tensor float32
compression_bin Text string
coverage Tensor float32
coverage_bin Text string
date Text string
density Tensor float32
density_bin Text string
summary Text string
text Text string
title Text string
url Text string
  • Citation:
@article{Grusky_2018,
   title={Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies},
   url={http://dx.doi.org/10.18653/v1/n18-1065},
   DOI={10.18653/v1/n18-1065},
   journal={Proceedings of the 2018 Conference of the North American Chapter of
          the Association for Computational Linguistics: Human Language
          Technologies, Volume 1 (Long Papers)},
   publisher={Association for Computational Linguistics},
   author={Grusky, Max and Naaman, Mor and Artzi, Yoav},
   year={2018}
}