参考：
abstract_narrative_understanding
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/abstract_narrative_understanding')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
3000 |
'train' |
2400 |
'validation' |
600 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
anachronisms
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/anachronisms')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
230 |
'train' |
184 |
'validation' |
46 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
analogical_similarity
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/analogical_similarity')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
323 |
'train' |
259 |
'validation' |
64 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
analytic_entailment
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/analytic_entailment')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
70 |
'train' |
54 |
'validation' |
16 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
arithmetic
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/arithmetic')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
15023 |
'train' |
12019 |
'validation' |
3004 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
ascii_word_recognition
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/ascii_word_recognition')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
5000 |
'train' |
4000 |
'validation' |
1000 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
authorship_verification
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/authorship_verification')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
880 |
'train' |
704 |
'validation' |
176 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
auto_categorization
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/auto_categorization')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
328 |
'train' |
263 |
'validation' |
65 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
auto_debugging
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/auto_debugging')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
34 |
'train' |
18 |
'validation' |
16 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
bbq_lite_json
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/bbq_lite_json')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
16076 |
'train' |
12866 |
'validation' |
3210 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
bridging_anaphora_resolution_barqa
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/bridging_anaphora_resolution_barqa')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
648 |
'train' |
519 |
'validation' |
129 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
causal_judgment
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/causal_judgment')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
190 |
'train' |
152 |
'validation' |
38 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
cause_and_effect
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/cause_and_effect')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
153 |
'train' |
123 |
'validation' |
30 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
checkmate_in_one
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/checkmate_in_one')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
3498 |
'train' |
2799 |
'validation' |
699 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
chess_state_tracking
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/chess_state_tracking')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
6000 |
'train' |
4800 |
'validation' |
1200 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
chinese_remainder_theorem
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/chinese_remainder_theorem')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
500 |
'train' |
400 |
'validation' |
100 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
cifar10_classification
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/cifar10_classification')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
20000 |
'train' |
16000 |
'validation' |
4000 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
code_line_description
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/code_line_description')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
60 |
'train' |
44 |
'validation' |
16 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
codenames
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/codenames')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
85 |
'train' |
68 |
'validation' |
17 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
color
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/color')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
4000 |
'train' |
3200 |
'validation' |
800 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
common_morpheme
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/common_morpheme')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
50 |
'train' |
34 |
'validation' |
16 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
conceptual_combinations
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/conceptual_combinations')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
103 |
'train' |
84 |
'validation' |
19 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
conlang_translation
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/conlang_translation')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
164 |
'train' |
132 |
'validation' |
32 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
contextual_parametric_knowledge_conflicts
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/contextual_parametric_knowledge_conflicts')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
17528 |
'train' |
14023 |
'validation' |
3505 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
crash_blossom
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/crash_blossom')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
38 |
'train' |
22 |
'validation' |
16 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
crass_ai
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/crass_ai')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
44 |
'train' |
28 |
'validation' |
16 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
cryobiology_spanish
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/cryobiology_spanish')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
146 |
'train' |
117 |
'validation' |
29 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
cryptonite
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/cryptonite')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
26157 |
'train' |
20926 |
'validation' |
5231 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
cs_algorithms
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/cs_algorithms')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
1320 |
'train' |
1056 |
'validation' |
264 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
dark_humor_detection
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/dark_humor_detection')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
80 |
'train' |
64 |
'validation' |
16 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
date_understanding
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/date_understanding')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
369 |
'train' |
296 |
'validation' |
73 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
disambiguation_qa
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/disambiguation_qa')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
258 |
'train' |
207 |
'validation' |
51 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
discourse_marker_prediction
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/discourse_marker_prediction')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
857 |
'train' |
686 |
'validation' |
171 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
disfl_qa
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/disfl_qa')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
8000 |
'train' |
6400 |
'validation' |
1600 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
dyck_languages
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/dyck_languages')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
1000 |
'train' |
800 |
'validation' |
200 |
- Features:
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
elementary_math_qa
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/elementary_math_qa')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
38160 |
'train' |
30531 |
'validation' |
7629 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
emoji_movie
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/emoji_movie')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
100 |
'train' |
80 |
'validation' |
20 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
emojis_emotion_prediction
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/emojis_emotion_prediction')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
131 |
'train' |
105 |
'validation' |
26 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
empirical_judgments
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/empirical_judgments')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
99 |
'train' |
80 |
'validation' |
19 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
english_proverbs
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/english_proverbs')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
34 |
'train' |
18 |
'validation' |
16 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
english_russian_proverbs
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/english_russian_proverbs')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
80 |
'train' |
64 |
'validation' |
16 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
entailed_polarity
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/entailed_polarity')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
148 |
'train' |
119 |
'validation' |
29 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
entailed_polarity_hindi
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/entailed_polarity_hindi')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
138 |
'train' |
111 |
'validation' |
27 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
epistemic_reasoning
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/epistemic_reasoning')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
2000 |
'train' |
1600 |
'validation' |
400 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
evaluating_information_essentiality
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/evaluating_information_essentiality')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
68 |
'train' |
52 |
'validation' |
16 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
fact_checker
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/fact_checker')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
7154 |
'train' |
5724 |
'validation' |
1430 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
fantasy_reasoning
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/fantasy_reasoning')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
201 |
'train' |
161 |
'validation' |
40 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
few_shot_nlg
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/few_shot_nlg')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
153 |
'train' |
123 |
'validation' |
30 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
figure_of_speech_detection
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/figure_of_speech_detection')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
59 |
'train' |
43 |
'validation' |
16 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
formal_fallacies_syllogisms_negation
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/formal_fallacies_syllogisms_negation')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
14200 |
'train' |
11360 |
'validation' |
2840 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
gem
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/gem')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
14802 |
'train' |
11845 |
'validation' |
2957 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
gender_inclusive_sentences_german
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/gender_inclusive_sentences_german')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
200 |
'train' |
160 |
'validation' |
40 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
general_knowledge
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/general_knowledge')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
70 |
'train' |
54 |
'validation' |
16 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
geometric_shapes
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/geometric_shapes')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
359 |
'train' |
288 |
'validation' |
71 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
goal_step_wikihow
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/goal_step_wikihow')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
7053 |
'train' |
5643 |
'validation' |
1410 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
gre_reading_comprehension
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/gre_reading_comprehension')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
31 |
'train' |
15 |
'validation' |
16 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
hhh_alignment
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/hhh_alignment')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
221 |
'train' |
179 |
'validation' |
42 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
hindi_question_answering
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/hindi_question_answering')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
6610 |
'train' |
5288 |
'validation' |
1322 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
hindu_knowledge
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/hindu_knowledge')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
175 |
'train' |
140 |
'validation' |
35 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
hinglish_toxicity
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/hinglish_toxicity')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
200 |
'train' |
160 |
'validation' |
40 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
human_organs_senses
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/human_organs_senses')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
42 |
'train' |
26 |
'validation' |
16 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
hyperbaton
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/hyperbaton')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
50000 |
'train' |
40000 |
'validation' |
10000 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
identify_math_theorems
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/identify_math_theorems')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
53 |
'train' |
37 |
'validation' |
16 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
identify_odd_metaphor
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/identify_odd_metaphor')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
47 |
'train' |
31 |
'validation' |
16 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
implicatures
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/implicatures')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
492 |
'train' |
394 |
'validation' |
98 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
implicit_relations
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/implicit_relations')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
85 |
'train' |
68 |
'validation' |
17 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
intent_recognition
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/intent_recognition')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
693 |
'train' |
555 |
'validation' |
138 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
international_phonetic_alphabet_nli
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/international_phonetic_alphabet_nli')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
126 |
'train' |
101 |
'validation' |
25 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
international_phonetic_alphabet_transliterate
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/international_phonetic_alphabet_transliterate')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
1003 |
'train' |
803 |
'validation' |
200 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
intersect_geometry
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/intersect_geometry')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
249999 |
'train' |
200000 |
'validation' |
49999 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_targets": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"multiple_choice_scores": {
"feature": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
irony_identification
使用以下命令在 TFDS 中加载此数据集：
ds = tfds.load('huggingface:bigbench/irony_identification')
- 说明：
The Beyond the Imitation Game Benchmark (BIG-bench) is a collaborative benchmark intended to
probe large language models, and extrapolate their future capabilities.
- 许可：Apache License 2.0
- 版本：0.0.0
- 拆分：
| 拆分 | 样本 |
|---|---|
'default' |
99 |
'train' |
80 |
'validation' |
19 |
- 特征：
{
"idx": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"inputs": {
"dtype": "string",
"id": null,
"_type": "Value"