Class SplitGenerator

Defined in core/

Defines the split information for the generator.

This should be used as returned value of GeneratorBasedBuilder._split_generators. See GeneratorBasedBuilder._split_generators for more info and example of usage.



Constructs a SplitGenerator.


  • name: str or list<str>, name of the Split for which the generator will create the examples. If a list is given, the generator examples will be distributed among the splits proportionally to the num_shards.
  • num_shards: int or list<int>, number of shards between which the generated examples will be written. If name is a list, then num_shards should be a list with the same number of elements.
  • gen_kwargs: dict, kwargs to forward to the _generate_examples() method of the builder.