tfds.core.ReadInstruction

View source on GitHub

Class ReadInstruction

Reading instruction for a dataset.

Examples of usage:

# The following lines are equivalent:
ds = tfds.load('mnist', split='test[:33%]')
ds = tfds.load('mnist', split=ReadInstruction.from_spec('test[:33%]'))
ds = tfds.load('mnist', split=ReadInstruction('test', to=33, unit='%'))
ds = tfds.load('mnist', split=ReadInstruction(
    'test', from_=0, to=33, unit='%'))

# The following lines are equivalent:
ds = tfds.load('mnist', split='test[:33%]+train[1:-1]')
ds = tfds.load('mnist', split=ReadInstruction.from_spec(
    'test[:33%]+train[1:-1]'))
ds = tfds.load('mnist', split=(
    ReadInstruction.('test', to=33, unit='%') +
    ReadInstruction.('train', from_=1, to=-1, unit='abs')))

# 10-fold validation:
tests = tfds.load(
    'mnist',
    [ReadInstruction('train', from_=k, to=k+10, unit='%')
     for k in range(0, 100, 10)])
trains = tfds.load(
    'mnist',
    [RI('train', to=k, unit='%') + RI('train', from_=k+10, unit='%')
     for k in range(0, 100, 10)])

__init__

View source

__init__(
    split_name,
    rounding='closest',
    from_=None,
    to=None,
    unit=None
)

Initialize ReadInstruction.

Args:

split_name (str): name of the split to read. Eg: 'train'. rounding (str): The rounding behaviour to use when percent slicing is used. Ignored when slicing with absolute indices. Possible values: - 'closest' (default): The specified percentages are rounded to the closest value. Use this if you want specified percents to be as much exact as possible. - 'pct1dropremainder': the specified percentages are treated as multiple of 1%. Use this option if you want consistency. Eg: len(5%) == 5 * len(1%). Using this option, one might not be able to use the full set of examples, if the number of those is not a multiple of 100. from (int): to (int): alternative way of specifying slicing boundaries. If any of {from_, to, unit} argument is used, slicing cannot be specified as string. unit (str): optional, one of: '%': to set the slicing unit as percents of the split size. 'abs': to set the slicing unit as absolute numbers.

Methods

__add__

View source

__add__(other)

Returns a new ReadInstruction obj, result of appending other to self.

from_spec

View source

@classmethod
from_spec(
    cls,
    spec
)

Creates a ReadInstruction instance out of a string spec.

Args:

spec (str): split(s) + optional slice(s) to read. A slice can be specified, using absolute numbers (int) or percentages (int). E.g. test: test split. test + validation: test split + validation split. test[10:]: test split, minus its first 10 records. test[:10%]: first 10% records of test split. test[:-5%]+train[40%:60%]: first 95% of test + middle 20% of train.

Returns:

ReadInstruction instance.

to_absolute

View source

to_absolute(name2len)

Translate instruction into a list of absolute instructions.

Those absolute instructions are then to be added together.

Args:

• name2len: dict associating split names to number of examples.

Returns:

list of _AbsoluteInstruction instances (corresponds to the + in spec).