Apply to speak at TensorFlow World. Deadline April 23rd. Propose talk

tfds.core.NamedSplit

Class NamedSplit

Inherits From: SplitBase

Defined in core/splits.py.

Descriptor corresponding to a named split (train, test, ...).

Each descriptor can be composed with other using addition or slice. Ex:

split = tfds.Split.TRAIN.subsplit(tfds.percent[0:25]) + tfds.Split.TEST

The resulting split will correspond to 25% of the train split merged with 100% of the test split.

split = (
    tfds.Split.TRAIN.subsplit(tfds.percent[:25]) +
    tfds.Split.TRAIN.subsplit(tfds.percent[75:])
)  # Error
split = tfds.Split.TEST + tfds.Split.ALL  # Error
split = (
    tfds.Split.TRAIN.subsplit(tfds.percent[:25]) +
    tfds.Split.TEST.subsplit(tfds.percent[:50])
)
split = (tfds.Split.TRAIN + tfds.Split.TEST).subsplit(tfds.percent[:50])

But not:

train = tfds.Split.TRAIN
test = tfds.Split.TEST
split = train.subsplit(tfds.percent[:25]).subsplit(tfds.percent[:25])
split = (train.subsplit(tfds.percent[:25]) + test).subsplit(tfds.percent[:50])

__init__

__init__(name)

Methods

__add__

__add__(other)

Merging: tfds.Split.TRAIN + tfds.Split.TEST.

__eq__

__eq__(other)

Equality: tfds.Split.TRAIN == 'train'.

get_read_instruction

get_read_instruction(split_dict)

subsplit

subsplit(
    arg=None,
    k=None,
    percent=None,
    weighted=None
)

Divides this split into subsplits.

There are 3 ways to define subsplits, which correspond to the 3 arguments k (get k even subsplits), percent (get a slice of the dataset with tfds.percent), and weighted (get subsplits with proportions specified by weighted).

Examples:

# 50% train, 50% test
train, test = split.subsplit(k=2)
# 50% train, 25% test, 25% validation
train, test, validation = split.subsplit(weighted=[2, 1, 1])
# Extract last 20%
subsplit = split.subsplit(tfds.percent[-20:])
train, test, valid = split.subsplit(k=3)  # 33%, 33%, 34%
s1, s2, s3, s4 = split.subsplit(weighted=[2, 2, 1, 1])  # 33%, 33%, 16%, 18%

Args:

  • arg: If no kwargs are given, arg will be interpreted as one of k, percent, or weighted depending on the type. For example:

    split.subsplit(10)  # Equivalent to split.subsplit(k=10)
    split.subsplit(tfds.percent[:-20])  # percent=tfds.percent[:-20]
    split.subsplit([1, 1, 2])  # weighted=[1, 1, 2]
    
  • k: int If set, subdivide the split into k equal parts.

  • percent: tfds.percent slice, return a single subsplit corresponding to a slice of the original split. For example: split.subsplit(tfds.percent[-20:]) # Last 20% of the dataset.

  • weighted: list[int], return a list of subsplits whose proportions match the normalized sum of the list. For example: split.subsplit(weighted=[1, 1, 2]) # 25%, 25%, 50%.

Returns:

A subsplit or list of subsplits extracted from this split object.