para_crawl

  • Description:

Web-Scale Parallel Corpora for Official European Languages.

@misc {paracrawl,
    title  = "ParaCrawl",
    year   = "2018",
    url    = "http://paracrawl.eu/download.html."
}

para_crawl/enbg (default config)

  • Config description: Translation dataset from English to bg.

  • Download size: 98.94 MiB

  • Dataset size: 362.46 MiB

  • Auto-cached (documentation): No

  • Splits:

Split Examples
'train' 1,039,885
  • Features:
Translation({
    'bg': Text(shape=(), dtype=tf.string),
    'en': Text(shape=(), dtype=tf.string),
})

para_crawl/encs

  • Config description: Translation dataset from English to cs.

  • Download size: 187.31 MiB

  • Dataset size: 666.34 MiB

  • Auto-cached (documentation): No

  • Splits:

Split Examples
'train' 2,981,949
  • Features:
Translation({
    'cs': Text(shape=(), dtype=tf.string),
    'en': Text(shape=(), dtype=tf.string),
})

para_crawl/enda

  • Config description: Translation dataset from English to da.

  • Download size: 174.34 MiB

  • Dataset size: 619.77 MiB

  • Auto-cached (documentation): No

  • Splits:

Split Examples
'train' 2,414,895
  • Features:
Translation({
    'da': Text(shape=(), dtype=tf.string),
    'en': Text(shape=(), dtype=tf.string),
})

para_crawl/ende

  • Config description: Translation dataset from English to de.

  • Download size: 1.22 GiB

  • Dataset size: 4.04 GiB

  • Auto-cached (documentation): No

  • Splits:

Split Examples
'train' 16,264,448
  • Features:
Translation({
    'de': Text(shape=(), dtype=tf.string),
    'en': Text(shape=(), dtype=tf.string),
})

para_crawl/enel

  • Config description: Translation dataset from English to el.

  • Download size: 184.59 MiB

  • Dataset size: 698.75 MiB

  • Auto-cached (documentation): No

  • Splits:

Split Examples
'train' 1,985,233
  • Features:
Translation({
    'el': Text(shape=(), dtype=tf.string),
    'en': Text(shape=(), dtype=tf.string),
})

para_crawl/enes

  • Config description: Translation dataset from English to es.

  • Download size: 1.82 GiB

  • Dataset size: 6.23 GiB

  • Auto-cached (documentation): No

  • Splits:

Split Examples
'train' 21,987,267
  • Features:
Translation({
    'en': Text(shape=(), dtype=tf.string),
    'es': Text(shape=(), dtype=tf.string),
})

para_crawl/enet

  • Config description: Translation dataset from English to et.

  • Download size: 66.91 MiB

  • Dataset size: 209.16 MiB

  • Auto-cached (documentation): Only when shuffle_files=False (train)

  • Splits:

Split Examples
'train' 853,422
  • Features:
Translation({
    'en': Text(shape=(), dtype=tf.string),
    'et': Text(shape=(), dtype=tf.string),
})

para_crawl/enfi

  • Config description: Translation dataset from English to fi.

  • Download size: 151.83 MiB

  • Dataset size: 543.85 MiB

  • Auto-cached (documentation): No

  • Splits:

Split Examples
'train' 2,156,069
  • Features:
Translation({
    'en': Text(shape=(), dtype=tf.string),
    'fi': Text(shape=(), dtype=tf.string),
})

para_crawl/enfr

  • Config description: Translation dataset from English to fr.

  • Download size: 2.63 GiB

  • Dataset size: 9.04 GiB

  • Auto-cached (documentation): No

  • Splits:

Split Examples
'train' 31,374,161
  • Features:
Translation({
    'en': Text(shape=(), dtype=tf.string),
    'fr': Text(shape=(), dtype=tf.string),
})

para_crawl/enga

  • Config description: Translation dataset from English to ga.

  • Download size: 28.03 MiB

  • Dataset size: 107.09 MiB

  • Auto-cached (documentation): Yes

  • Splits:

Split Examples
'train' 357,399
  • Features:
Translation({
    'en': Text(shape=(), dtype=tf.string),
    'ga': Text(shape=(), dtype=tf.string),
})

para_crawl/enhr

  • Config description: Translation dataset from English to hr.

  • Download size: 80.97 MiB

  • Dataset size: 256.37 MiB

  • Auto-cached (documentation): No

  • Splits:

Split Examples
'train' 1,002,053
  • Features:
Translation({
    'en': Text(shape=(), dtype=tf.string),
    'hr': Text(shape=(), dtype=tf.string),
})

para_crawl/enhu

  • Config description: Translation dataset from English to hu.

  • Download size: 114.24 MiB

  • Dataset size: 421.40 MiB

  • Auto-cached (documentation): No

  • Splits:

Split Examples
'train' 1,901,342
  • Features:
Translation({
    'en': Text(shape=(), dtype=tf.string),
    'hu': Text(shape=(), dtype=tf.string),
})

para_crawl/enit

  • Config description: Translation dataset from English to it.

  • Download size: 1017.30 MiB

  • Dataset size: 3.36 GiB

  • Auto-cached (documentation): No

  • Splits:

Split Examples
'train' 12,162,239
  • Features:
Translation({
    'en': Text(shape=(), dtype=tf.string),
    'it': Text(shape=(), dtype=tf.string),
})

para_crawl/enlt

  • Config description: Translation dataset from English to lt.

  • Download size: 63.28 MiB

  • Dataset size: 204.70 MiB

  • Auto-cached (documentation): Only when shuffle_files=False (train)

  • Splits:

Split Examples
'train' 844,643
  • Features:
Translation({
    'en': Text(shape=(), dtype=tf.string),
    'lt': Text(shape=(), dtype=tf.string),
})

para_crawl/enlv

  • Config description: Translation dataset from English to lv.

  • Download size: 45.17 MiB

  • Dataset size: 147.09 MiB

  • Auto-cached (documentation): Only when shuffle_files=False (train)

  • Splits:

Split Examples
'train' 553,060
  • Features:
Translation({
    'en': Text(shape=(), dtype=tf.string),
    'lv': Text(shape=(), dtype=tf.string),
})

para_crawl/enmt

  • Config description: Translation dataset from English to mt.

  • Download size: 18.15 MiB

  • Dataset size: 54.36 MiB

  • Auto-cached (documentation): Yes

  • Splits:

Split Examples
'train' 195,502
  • Features:
Translation({
    'en': Text(shape=(), dtype=tf.string),
    'mt': Text(shape=(), dtype=tf.string),
})

para_crawl/ennl

  • Config description: Translation dataset from English to nl.

  • Download size: 400.63 MiB

  • Dataset size: 1.40 GiB

  • Auto-cached (documentation): No

  • Splits:

Split Examples
'train' 5,659,268
  • Features:
Translation({
    'en': Text(shape=(), dtype=tf.string),
    'nl': Text(shape=(), dtype=tf.string),
})

para_crawl/enpl

  • Config description: Translation dataset from English to pl.

  • Download size: 257.90 MiB

  • Dataset size: 885.63 MiB

  • Auto-cached (documentation): No

  • Splits:

Split Examples
'train' 3,503,276
  • Features:
Translation({
    'en': Text(shape=(), dtype=tf.string),
    'pl': Text(shape=(), dtype=tf.string),
})

para_crawl/enpt

  • Config description: Translation dataset from English to pt.

  • Download size: 608.62 MiB

  • Dataset size: 2.05 GiB

  • Auto-cached (documentation): No

  • Splits:

Split Examples
'train' 8,141,940
  • Features:
Translation({
    'en': Text(shape=(), dtype=tf.string),
    'pt': Text(shape=(), dtype=tf.string),
})

para_crawl/enro

  • Config description: Translation dataset from English to ro.

  • Download size: 153.24 MiB

  • Dataset size: 534.34 MiB

  • Auto-cached (documentation): No

  • Splits:

Split Examples
'train' 1,952,043
  • Features:
Translation({
    'en': Text(shape=(), dtype=tf.string),
    'ro': Text(shape=(), dtype=tf.string),
})

para_crawl/ensk

  • Config description: Translation dataset from English to sk.

  • Download size: 96.61 MiB

  • Dataset size: 352.91 MiB

  • Auto-cached (documentation): No

  • Splits:

Split Examples
'train' 1,591,831
  • Features:
Translation({
    'en': Text(shape=(), dtype=tf.string),
    'sk': Text(shape=(), dtype=tf.string),
})

para_crawl/ensl

  • Config description: Translation dataset from English to sl.

  • Download size: 62.02 MiB

  • Dataset size: 187.66 MiB

  • Auto-cached (documentation): Only when shuffle_files=False (train)

  • Splits:

Split Examples
'train' 660,161
  • Features:
Translation({
    'en': Text(shape=(), dtype=tf.string),
    'sl': Text(shape=(), dtype=tf.string),
})

para_crawl/ensv

  • Config description: Translation dataset from English to sv.

  • Download size: 262.76 MiB

  • Dataset size: 905.72 MiB

  • Auto-cached (documentation): No

  • Splits:

Split Examples
'train' 3,476,729
  • Features:
Translation({
    'en': Text(shape=(), dtype=tf.string),
    'sv': Text(shape=(), dtype=tf.string),
})