hellaswag

  • Description:

The HellaSwag dataset is a benchmark for Commonsense NLI. It includes a context and some endings which complete the context.

  • Homepage: https://rowanzellers.com/hellaswag/

  • Source code: tfds.text.Hellaswag

  • Versions:

    • 0.0.1: No release notes.
    • 1.0.0: Adding separate splits for in-domain and out-of-domain validation/test sets.
    • 1.1.0 (default): Another split dimension for source (wikihow vs activitynet)
  • Download size: 68.18 MiB

  • Dataset size: 107.45 MiB

  • Auto-cached (documentation): Yes

  • Splits:

Split Examples
'test' 10,003
'test_ind_activitynet' 1,870
'test_ind_wikihow' 3,132
'test_ood_activitynet' 1,651
'test_ood_wikihow' 3,350
'train' 39,905
'train_activitynet' 14,740
'train_wikihow' 25,165
'validation' 10,042
'validation_ind_activitynet' 1,809
'validation_ind_wikihow' 3,192
'validation_ood_activitynet' 1,434
'validation_ood_wikihow' 3,607
  • Feature structure:
FeaturesDict({
    'activity_label': Text(shape=(), dtype=tf.string),
    'context': Text(shape=(), dtype=tf.string),
    'endings': Sequence(Text(shape=(), dtype=tf.string)),
    'label': tf.int32,
    'source_id': Text(shape=(), dtype=tf.string),
    'split_type': Text(shape=(), dtype=tf.string),
})
  • Feature documentation:
Feature Class Shape Dtype Description
FeaturesDict
activity_label Text tf.string
context Text tf.string
endings Sequence(Text) (None,) tf.string
label Tensor tf.int32
source_id Text tf.string
split_type Text tf.string
  • Citation:
@inproceedings{zellers2019hellaswag,
    title={HellaSwag: Can a Machine Really Finish Your Sentence?},
    author={Zellers, Rowan and Holtzman, Ari and Bisk, Yonatan and Farhadi, Ali and Choi, Yejin},
    booktitle ={Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
    year={2019}
}