• Description:

Corr2cause

Causal inference is one of the hallmarks of human intelligence.

Corr2cause is a large-scale dataset of more than 400K samples, on which seventeen existing LLMs are evaluated in the related paper.

Overall, Corr2cause contains 415,944 samples, with 18.57% in valid samples. The average length of the premise is 424.11 tokens, and hypothesis 10.83 tokens. The data is split into 411,452 training samples, 2,246 development and test samples, respectively. Since the main purpose of the dataset is to benchmark the performance of LLMs, the test and development sets have been prioritized to have a comprehensive coverage over all sizes of graphs.

• Homepage: https://github.com/causalNLP/corr2cause/tree/main

• Source code: tfds.datasets.corr2cause.Builder

• Versions:

  • 1.0.0 (default): Initial release.

• Download size: 727.22 MiB

• Dataset size: 739.91 MiB

• Auto-cached (documentation): No

• Splits:

• Feature structure:

FeaturesDict({
    'input': Text(shape=(), dtype=string),
    'label': int64,
})

• Feature documentation:

• Supervised keys (See as_supervised doc): None

• Figure (tfds.show_examples): Not supported.

• Examples (tfds.as_dataframe):

• Citation:

@misc{jin2023large,
      title={Can Large Language Models Infer Causation from Correlation?},
      author={Zhijing Jin and Jiarui Liu and Zhiheng Lyu and Spencer Poff and Mrinmaya Sachan and Rada Mihalcea and Mona Diab and Bernhard Schölkopf},
      year={2023},
      eprint={2306.05836},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}