para_crawl

References:

• Code
• Huggingface

enbg

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/enbg')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	1039885

• Features:

{
    "translation": {
        "languages": [
            "en",
            "bg"
        ],
        "id": null,
        "_type": "Translation"
    }
}

encs

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/encs')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	2981949

• Features:

{
    "translation": {
        "languages": [
            "en",
            "cs"
        ],
        "id": null,
        "_type": "Translation"
    }
}

enda

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/enda')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	2414895

• Features:

{
    "translation": {
        "languages": [
            "en",
            "da"
        ],
        "id": null,
        "_type": "Translation"
    }
}

ende

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/ende')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	16264448

• Features:

{
    "translation": {
        "languages": [
            "en",
            "de"
        ],
        "id": null,
        "_type": "Translation"
    }
}

enel

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/enel')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	1985233

• Features:

{
    "translation": {
        "languages": [
            "en",
            "el"
        ],
        "id": null,
        "_type": "Translation"
    }
}

enes

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/enes')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	21987267

• Features:

{
    "translation": {
        "languages": [
            "en",
            "es"
        ],
        "id": null,
        "_type": "Translation"
    }
}

enet

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/enet')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	853422

• Features:

{
    "translation": {
        "languages": [
            "en",
            "et"
        ],
        "id": null,
        "_type": "Translation"
    }
}

enfi

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/enfi')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	2156069

• Features:

{
    "translation": {
        "languages": [
            "en",
            "fi"
        ],
        "id": null,
        "_type": "Translation"
    }
}

enfr

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/enfr')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	31374161

• Features:

{
    "translation": {
        "languages": [
            "en",
            "fr"
        ],
        "id": null,
        "_type": "Translation"
    }
}

enga

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/enga')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	357399

• Features:

{
    "translation": {
        "languages": [
            "en",
            "ga"
        ],
        "id": null,
        "_type": "Translation"
    }
}

enhr

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/enhr')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	1002053

• Features:

{
    "translation": {
        "languages": [
            "en",
            "hr"
        ],
        "id": null,
        "_type": "Translation"
    }
}

enhu

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/enhu')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	1901342

• Features:

{
    "translation": {
        "languages": [
            "en",
            "hu"
        ],
        "id": null,
        "_type": "Translation"
    }
}

enit

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/enit')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	12162239

• Features:

{
    "translation": {
        "languages": [
            "en",
            "it"
        ],
        "id": null,
        "_type": "Translation"
    }
}

enlt

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/enlt')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	844643

• Features:

{
    "translation": {
        "languages": [
            "en",
            "lt"
        ],
        "id": null,
        "_type": "Translation"
    }
}

enlv

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/enlv')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	553060

• Features:

{
    "translation": {
        "languages": [
            "en",
            "lv"
        ],
        "id": null,
        "_type": "Translation"
    }
}

enmt

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/enmt')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	195502

• Features:

{
    "translation": {
        "languages": [
            "en",
            "mt"
        ],
        "id": null,
        "_type": "Translation"
    }
}

ennl

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/ennl')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	5659268

• Features:

{
    "translation": {
        "languages": [
            "en",
            "nl"
        ],
        "id": null,
        "_type": "Translation"
    }
}

enpl

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/enpl')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	3503276

• Features:

{
    "translation": {
        "languages": [
            "en",
            "pl"
        ],
        "id": null,
        "_type": "Translation"
    }
}

enpt

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/enpt')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	8141940

• Features:

{
    "translation": {
        "languages": [
            "en",
            "pt"
        ],
        "id": null,
        "_type": "Translation"
    }
}

enro

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/enro')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	1952043

• Features:

{
    "translation": {
        "languages": [
            "en",
            "ro"
        ],
        "id": null,
        "_type": "Translation"
    }
}

ensk

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/ensk')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	1591831

• Features:

{
    "translation": {
        "languages": [
            "en",
            "sk"
        ],
        "id": null,
        "_type": "Translation"
    }
}

ensl

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/ensl')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	660161

• Features:

{
    "translation": {
        "languages": [
            "en",
            "sl"
        ],
        "id": null,
        "_type": "Translation"
    }
}

ensv

Use the following command to load this dataset in TFDS:

ds = tfds.load('huggingface:para_crawl/ensv')

• Description:

Web-Scale Parallel Corpora for Official European Languages.

• License: No known license
• Version: 1.0.0
• Splits:

Split	Examples
'train'	3476729

• Features:

{
    "translation": {
        "languages": [
            "en",
            "sv"
        ],
        "id": null,
        "_type": "Translation"
    }
}