<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/sequential_writer.py#L187-L369) |

Class to write a TFDS dataset sequentially.

    tfds.core.SequentialWriter(
        ds_info: dataset_info.DatasetInfo,
        max_examples_per_shard: int,
        overwrite: bool = True,
        file_format: str = 'tfrecord'
    )

The SequentialWriter can be used to generate TFDS datasets by directly
appending TF Examples to the desired splits.

Once the user creates a SequentialWriter with a given DatasetInfo, they can
create splits, append examples to them, and close them whenever they are
finished.

#### Note that:

- Not closing a split may cause data to be lost.
- The examples are written to disk in the same order that they are given to the writer.
- Since the SequentialWriter doesn't know how many examples are going to be written, it can't estimate the optimal number of shards per split. Use the `max_examples_per_shard` parameter in the constructor to control how many elements there should be per shard.

The datasets written with this writer can be read directly with
[`tfds.builder_from_directories`](https://www.tensorflow.org/datasets/api_docs/python/tfds/builder_from_directories).

#### Example:

writer = SequentialWriter(ds_info=ds_info, max_examples_per_shard=1000)
writer.initialize_splits(\['train', 'test'\])

while (...):
# Code that generates the examples
writer.add_examples({'train': \[example1, example2\],
'test': \[example3\]})
...

writer.close_splits()

<br />

<br />

<br />

| ## Args ||
|---|---|
| `ds_info` | DatasetInfo for this dataset. |
| `max_examples_per_shard` | maximum number of examples to write per shard. |
| `overwrite` | if True, it ignores and overwrites any existing data. Otherwise, it loads the existing dataset and appends the new data (new data will always be created as new shards). |
| `file_format` | An entry in file_adapters.FileFormat. |

<br />

## Methods

### `add_examples`

[View source](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/sequential_writer.py#L319-L342)

    add_examples(
        split_examples: Dict[str, List[Any]]
    ) -> None

Adds examples to the splits.

<br />

<br />

<br />

| Args ||
|---|---|
| `split_examples` | dictionary of `split_name`:list_of_examples that includes the list of examples that has to be added to each of the splits. Not all the existing splits have to be in the dictionary |

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `KeyError` | if any of the splits doesn't exist. |

<br />

### `close_all`

[View source](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/sequential_writer.py#L365-L369)

    close_all() -> None

Closes all the open splits.

### `close_splits`

[View source](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/sequential_writer.py#L349-L363)

    close_splits(
        splits: List[str]
    ) -> None

Closes the given list of splits.

<br />

<br />

<br />

| Args ||
|---|---|
| `splits` | list of split names. |

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `KeyError` | if any of the splits doesn't exist. |

<br />

### `initialize_splits`

[View source](https://github.com/tensorflow/datasets/blob/v4.9.10/tensorflow_datasets/core/sequential_writer.py#L292-L317)

    initialize_splits(
        splits: List[str], fail_if_exists: bool = True
    ) -> None

Adds new splits to the dataset.

<br />

<br />

<br />

| Args ||
|---|---|
| `splits` | list of split names to add. |
| `fail_if_exists` | will fail if this split already contains data. |

<br />

<br />

<br />

<br />

| Raises ||
|---|---|
| `KeyError` | if the split is already present. |

<br />