common_voice

Description:

Mozilla Common Voice Dataset

Additional Documentation: Explore on Papers With Code
Homepage: https://voice.mozilla.org/en/datasets
Source code: tfds.audio.CommonVoice
Versions:
- 1.0.0: Initial release.
- 2.0.0 (default): Updated to corpus 6.1 from 2020-12-11.
Feature structure:

FeaturesDict({
    'accent': Text(shape=(), dtype=string),
    'age': Text(shape=(), dtype=string),
    'client_id': Text(shape=(), dtype=string),
    'downvotes': Scalar(shape=(), dtype=int32),
    'gender': ClassLabel(shape=(), dtype=int64, num_classes=3),
    'segment': Text(shape=(), dtype=string),
    'sentence': Text(shape=(), dtype=string),
    'upvotes': Scalar(shape=(), dtype=int32),
    'voice': Audio(shape=(None,), dtype=int64),
})

Feature documentation:

Feature	Class	Shape	Dtype	Description
	FeaturesDict
accent	Text		string	Accent of the speaker, see https://github.com/common-voice/common-voice/blob/main/web/src/stores/demographics.ts
age	Text		string	Age bucket of the speaker (e.g. teens, or fourties), see https://github.com/common-voice/common-voice/blob/main/web/src/stores/demographics.ts
client_id	Text		string	Hashed UUID of a given user
downvotes	Scalar		int32	Number of people who said audio does not match text
gender	ClassLabel		int64	Gender of the speaker
segment	Text		string	If sentence belongs to a custom dataset segment, it will be listed here
sentence	Text		string	Supposed transcription of the audio
upvotes	Scalar		int32	Number of people who said audio matches the text
voice	Audio	(None,)	int64

Supervised keys (See as_supervised doc): None
Figure (tfds.show_examples): Not supported.
Citation:

common_voice/en (default config)

Config description: Language Code: en
Download size: 56.45 GiB
Dataset size: 2.79 TiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	16,164
`'test'`	16,164
`'train'`	564,337
`'validation'`	1,224,864

Examples (tfds.as_dataframe):

common_voice/ab

Config description: Language Code: ab
Download size: 39.14 MiB
Dataset size: 133.24 MiB
Auto-cached (documentation): Yes
Splits:

Split	Examples
`'test'`	9
`'train'`	22
`'validation'`	31

Examples (tfds.as_dataframe):

common_voice/ar

Config description: Language Code: ar
Download size: 1.64 GiB
Dataset size: 67.16 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	7,517
`'test'`	7,622
`'train'`	14,227
`'validation'`	43,291

Examples (tfds.as_dataframe):

common_voice/as

Config description: Language Code: as
Download size: 21.20 MiB
Dataset size: 1.65 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	124
`'test'`	110
`'train'`	270
`'validation'`	504

Examples (tfds.as_dataframe):

common_voice/br

Config description: Language Code: br
Download size: 443.72 MiB
Dataset size: 13.46 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	1,997
`'test'`	2,087
`'train'`	2,780
`'validation'`	8,560

Examples (tfds.as_dataframe):

common_voice/ca

Config description: Language Code: ca
Download size: 19.32 GiB
Dataset size: 1.19 TiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	15,724
`'test'`	15,724
`'train'`	285,584
`'validation'`	416,701

Examples (tfds.as_dataframe):

common_voice/cnh

Config description: Language Code: cnh
Download size: 153.86 MiB
Dataset size: 5.12 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	756
`'test'`	752
`'train'`	807
`'validation'`	2,432

Examples (tfds.as_dataframe):

common_voice/cs

Config description: Language Code: cs
Download size: 1.18 GiB
Dataset size: 56.89 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	4,118
`'test'`	4,144
`'train'`	5,655
`'validation'`	30,431

Examples (tfds.as_dataframe):

common_voice/cv

Config description: Language Code: cv
Download size: 418.98 MiB
Dataset size: 8.10 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	818
`'test'`	788
`'train'`	931
`'validation'`	3,496

Examples (tfds.as_dataframe):

common_voice/cy

Config description: Language Code: cy
Download size: 3.20 GiB
Dataset size: 128.68 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	4,776
`'test'`	4,820
`'train'`	6,839
`'validation'`	72,984

Examples (tfds.as_dataframe):

common_voice/de

Config description: Language Code: de
Download size: 21.68 GiB
Dataset size: 1.29 TiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	15,588
`'test'`	15,588
`'train'`	246,525
`'validation'`	565,186

Examples (tfds.as_dataframe):

common_voice/dv

Config description: Language Code: dv
Download size: 515.45 MiB
Dataset size: 31.59 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	2,077
`'test'`	2,202
`'train'`	2,680
`'validation'`	11,866

Examples (tfds.as_dataframe):

common_voice/el

Config description: Language Code: el
Download size: 363.89 MiB
Dataset size: 14.62 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	1,401
`'test'`	1,522
`'train'`	2,316
`'validation'`	5,996

Examples (tfds.as_dataframe):

common_voice/eo

Config description: Language Code: eo
Download size: 2.69 GiB
Dataset size: 167.14 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	8,987
`'test'`	8,969
`'train'`	19,587
`'validation'`	58,094

Examples (tfds.as_dataframe):

common_voice/es

Config description: Language Code: es
Download size: 15.08 GiB
Dataset size: 684.66 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	15,089
`'test'`	15,089
`'train'`	161,813
`'validation'`	236,314

Examples (tfds.as_dataframe):

common_voice/et

Config description: Language Code: et
Download size: 731.63 MiB
Dataset size: 37.95 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	2,507
`'test'`	2,509
`'train'`	2,966
`'validation'`	10,683

Examples (tfds.as_dataframe):

common_voice/eu

Config description: Language Code: eu
Download size: 3.41 GiB
Dataset size: 127.60 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	5,172
`'test'`	5,172
`'train'`	7,505
`'validation'`	63,009

Examples (tfds.as_dataframe):

common_voice/fa

Config description: Language Code: fa
Download size: 8.27 GiB
Dataset size: 328.61 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	5,213
`'test'`	5,213
`'train'`	7,593
`'validation'`	251,659

Examples (tfds.as_dataframe):

common_voice/fi

Config description: Language Code: fi
Download size: 47.57 MiB
Dataset size: 3.41 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	415
`'test'`	428
`'train'`	460
`'validation'`	1,305

Examples (tfds.as_dataframe):

common_voice/fr

Config description: Language Code: fr
Download size: 17.82 GiB
Dataset size: 1.17 TiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	15,763
`'test'`	15,763
`'train'`	298,982
`'validation'`	461,004

Examples (tfds.as_dataframe):

common_voice/fy-NL

Config description: Language Code: fy-NL
Download size: 1.15 GiB
Dataset size: 29.93 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	2,790
`'test'`	3,020
`'train'`	3,927
`'validation'`	10,495

Examples (tfds.as_dataframe):

common_voice/ga-IE

Config description: Language Code: ga-IE
Download size: 149.30 MiB
Dataset size: 5.11 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	497
`'test'`	506
`'train'`	541
`'validation'`	3,352

Examples (tfds.as_dataframe):

common_voice/hi

Config description: Language Code: hi
Download size: 20.43 MiB
Dataset size: 1.15 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	135
`'test'`	127
`'train'`	157
`'validation'`	419

Examples (tfds.as_dataframe):

common_voice/hsb

Config description: Language Code: hsb
Download size: 75.69 MiB
Dataset size: 5.67 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	172
`'test'`	387
`'train'`	808
`'validation'`	1,367

Examples (tfds.as_dataframe):

common_voice/hu

Config description: Language Code: hu
Download size: 231.51 MiB
Dataset size: 17.07 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	1,434
`'test'`	1,649
`'train'`	3,348
`'validation'`	6,457

Examples (tfds.as_dataframe):

common_voice/ia

Config description: Language Code: ia
Download size: 216.01 MiB
Dataset size: 14.99 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	1,601
`'test'`	899
`'train'`	3,477
`'validation'`	5,978

Examples (tfds.as_dataframe):

common_voice/id

Config description: Language Code: id
Download size: 453.87 MiB
Dataset size: 17.20 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	1,835
`'test'`	1,844
`'train'`	2,130
`'validation'`	8,696

Examples (tfds.as_dataframe):

common_voice/it

Config description: Language Code: it
Download size: 5.20 GiB
Dataset size: 316.38 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	12,928
`'test'`	12,928
`'train'`	58,015
`'validation'`	102,579

Examples (tfds.as_dataframe):

common_voice/ja

Config description: Language Code: ja
Download size: 145.80 MiB
Dataset size: 6.83 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	586
`'test'`	632
`'train'`	722
`'validation'`	3,072

Examples (tfds.as_dataframe):

common_voice/ka

Config description: Language Code: ka
Download size: 99.45 MiB
Dataset size: 7.51 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	527
`'test'`	656
`'train'`	1,058
`'validation'`	2,275

Examples (tfds.as_dataframe):

common_voice/kab

Config description: Language Code: kab
Download size: 15.99 GiB
Dataset size: 718.51 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	14,622
`'test'`	14,622
`'train'`	120,530
`'validation'`	573,718

Examples (tfds.as_dataframe):

common_voice/ky

Config description: Language Code: ky
Download size: 552.60 MiB
Dataset size: 18.70 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	1,511
`'test'`	1,503
`'train'`	1,955
`'validation'`	9,236

Examples (tfds.as_dataframe):

common_voice/lg

Config description: Language Code: lg
Download size: 198.55 MiB
Dataset size: 6.65 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	384
`'test'`	584
`'train'`	1,250
`'validation'`	2,220

Examples (tfds.as_dataframe):

common_voice/lt

Config description: Language Code: lt
Download size: 129.03 MiB
Dataset size: 4.79 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	244
`'test'`	466
`'train'`	931
`'validation'`	1,644

Examples (tfds.as_dataframe):

common_voice/lv

Config description: Language Code: lv
Download size: 198.66 MiB
Dataset size: 13.07 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	2,002
`'test'`	1,882
`'train'`	2,552
`'validation'`	6,444

Examples (tfds.as_dataframe):

common_voice/mn

Config description: Language Code: mn
Download size: 463.84 MiB
Dataset size: 22.09 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	1,837
`'test'`	1,862
`'train'`	2,183
`'validation'`	7,487

Examples (tfds.as_dataframe):

common_voice/mt

Config description: Language Code: mt
Download size: 405.42 MiB
Dataset size: 15.09 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	1,516
`'test'`	1,617
`'train'`	2,036
`'validation'`	5,747

Examples (tfds.as_dataframe):

common_voice/nl

Config description: Language Code: nl
Download size: 1.62 GiB
Dataset size: 90.20 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	4,938
`'test'`	5,708
`'train'`	9,460
`'validation'`	52,488

Examples (tfds.as_dataframe):

common_voice/or

Config description: Language Code: or
Download size: 189.85 MiB
Dataset size: 1.97 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	129
`'test'`	98
`'train'`	388
`'validation'`	615

Examples (tfds.as_dataframe):

common_voice/pa-IN

Config description: Language Code: pa-IN
Download size: 66.52 MiB
Dataset size: 1.03 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	44
`'test'`	116
`'train'`	211
`'validation'`	371

Examples (tfds.as_dataframe):

common_voice/pl

Config description: Language Code: pl
Download size: 3.29 GiB
Dataset size: 141.06 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	5,153
`'test'`	5,153
`'train'`	7,468
`'validation'`	90,791

Examples (tfds.as_dataframe):

common_voice/pt

Config description: Language Code: pt
Download size: 1.59 GiB
Dataset size: 75.64 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	4,592
`'test'`	4,641
`'train'`	6,514
`'validation'`	41,584

Examples (tfds.as_dataframe):

common_voice/rm-sursilv

Config description: Language Code: rm-sursilv
Download size: 263.17 MiB
Dataset size: 12.31 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	1,205
`'test'`	1,194
`'train'`	1,384
`'validation'`	3,783

Examples (tfds.as_dataframe):

common_voice/rm-vallader

Config description: Language Code: rm-vallader
Download size: 103.11 MiB
Dataset size: 4.89 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	357
`'test'`	378
`'train'`	574
`'validation'`	1,316

Examples (tfds.as_dataframe):

common_voice/ro

Config description: Language Code: ro
Download size: 249.84 MiB
Dataset size: 14.54 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	858
`'test'`	1,778
`'train'`	3,399
`'validation'`	6,039

Examples (tfds.as_dataframe):

common_voice/ru

Config description: Language Code: ru
Download size: 3.40 GiB
Dataset size: 175.04 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	7,963
`'test'`	8,007
`'train'`	15,481
`'validation'`	74,256

Examples (tfds.as_dataframe):

common_voice/rw

Config description: Language Code: rw
Download size: 39.62 GiB
Dataset size: 2.18 TiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	15,032
`'test'`	15,724
`'train'`	515,197
`'validation'`	832,929

Examples (tfds.as_dataframe):

common_voice/sah

Config description: Language Code: sah
Download size: 172.85 MiB
Dataset size: 9.42 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	405
`'test'`	757
`'train'`	1,442
`'validation'`	2,606

Examples (tfds.as_dataframe):

common_voice/sl

Config description: Language Code: sl
Download size: 212.43 MiB
Dataset size: 9.67 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	556
`'test'`	881
`'train'`	2,038
`'validation'`	4,669

Examples (tfds.as_dataframe):

common_voice/sv-SE

Config description: Language Code: sv-SE
Download size: 401.91 MiB
Dataset size: 18.27 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	2,019
`'test'`	2,027
`'train'`	2,331
`'validation'`	12,552

Examples (tfds.as_dataframe):

common_voice/ta

Config description: Language Code: ta
Download size: 648.28 MiB
Dataset size: 24.06 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	1,779
`'test'`	1,781
`'train'`	2,009
`'validation'`	12,652

Examples (tfds.as_dataframe):

common_voice/th

Config description: Language Code: th
Download size: 325.49 MiB
Dataset size: 18.32 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	1,922
`'test'`	2,188
`'train'`	2,917
`'validation'`	7,028

Examples (tfds.as_dataframe):

common_voice/tr

Config description: Language Code: tr
Download size: 592.09 MiB
Dataset size: 28.21 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	1,647
`'test'`	1,647
`'train'`	1,831
`'validation'`	18,685

Examples (tfds.as_dataframe):

common_voice/tt

Config description: Language Code: tt
Download size: 741.15 MiB
Dataset size: 46.85 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	2,127
`'test'`	4,485
`'train'`	11,211
`'validation'`	25,781

Examples (tfds.as_dataframe):

common_voice/uk

Config description: Language Code: uk
Download size: 1.13 GiB
Dataset size: 49.66 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	3,236
`'test'`	3,235
`'train'`	4,035
`'validation'`	22,337

Examples (tfds.as_dataframe):

common_voice/vi

Config description: Language Code: vi
Download size: 49.52 MiB
Dataset size: 1.47 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	200
`'test'`	198
`'train'`	221
`'validation'`	619

Examples (tfds.as_dataframe):

common_voice/vot

Config description: Language Code: vot
Download size: 7.43 MiB
Dataset size: 11.39 MiB
Auto-cached (documentation): Yes
Splits:

Split	Examples
`'train'`	3
`'validation'`	3

Examples (tfds.as_dataframe):

common_voice/zh-CN

Config description: Language Code: zh-CN
Download size: 2.03 GiB
Dataset size: 122.54 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	8,743
`'test'`	8,760
`'train'`	18,541
`'validation'`	36,405

Examples (tfds.as_dataframe):

common_voice/zh-HK

Config description: Language Code: zh-HK
Download size: 2.58 GiB
Dataset size: 78.80 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	5,172
`'test'`	5,172
`'train'`	7,506
`'validation'`	41,835

Examples (tfds.as_dataframe):

common_voice/zh-TW

Config description: Language Code: zh-TW
Download size: 2.03 GiB
Dataset size: 69.06 GiB
Auto-cached (documentation): No
Splits:

Split	Examples
`'dev'`	2,895
`'test'`	2,895
`'train'`	3,507
`'validation'`	61,232

Examples (tfds.as_dataframe):