tf.contrib.training.HParams

View source on GitHub

Class to hold a set of hyperparameters as name-value pairs.

A HParams object holds hyperparameters used to build and train a model, such as the number of hidden units in a neural net layer or the learning rate to use when training.

You first create a HParams object by specifying the names and values of the hyperparameters.

To make them easily accessible the parameter names are added as direct attributes of the class. A typical usage is as follows:

# Create a HParams object specifying names and values of the model
# hyperparameters:
hparams = HParams(learning_rate=0.1, num_hidden_units=100)

# The hyperparameter are available as attributes of the HParams object:
hparams.learning_rate ==> 0.1
hparams.num_hidden_units ==> 100

Hyperparameters have type, which is inferred from the type of their value passed at construction type. The currently supported types are: integer, float, boolean, string, and list of integer, float, boolean, or string.

You can override hyperparameter values by calling the parse() method, passing a string of comma separated name=value pairs. This is intended to make it possible to override any hyperparameter values from a single command-line flag to which the user passes 'hyper-param=value' pairs. It avoids having to define one flag for each hyperparameter.

The syntax expected for each value depends on the type of the parameter. See parse() for a description of the syntax.

Example:

# Define a command line flag to pass name=value pairs.
# For example using argparse:
import argparse
parser = argparse.ArgumentParser(description='Train my model.')
parser.add_argument('--hparams', type=str,
                    help='Comma separated list of "name=value" pairs.')
args = parser.parse_args()
...
def my_program():
  # Create a HParams object specifying the names and values of the
  # model hyperparameters:
  hparams = tf.contrib.training.HParams(
      learning_rate=0.1,
      num_hidden_units=100,
      activations=['relu', 'tanh'])

  # Override hyperparameters values by parsing the command line
  hparams.parse(args.hparams)

  # If the user passed `--hparams=learning_rate=0.3` on the command line
  # then 'hparams' has the following attributes:
  hparams.learning_rate ==> 0.3
  hparams.num_hidden_units ==> 100
  hparams.activations ==> ['relu', 'tanh']

  # If the hyperparameters are in json format use parse_json:
  hparams.parse_json('{"learning_rate": 0.3, "activations": "relu"}')

hparam_def Serialized hyperparameters, encoded as a hparam_pb2.HParamDef protocol buffer. If provided, this object is initialized by deserializing hparam_def. Otherwise **kwargs is used.
model_structure An instance of ModelStructure, defining the feature crosses to be used in the Trial.
**kwargs Key-value pairs where the key is the hyperparameter name and the value is the value for the parameter.

ValueError If both hparam_def and initialization values are provided, or if one of the arguments is invalid.

Methods

add_hparam

View source

Adds {name, value} pair to hyperparameters.

Args
name Name of the hyperparameter.
value Value of the hyperparameter. Can be one of the following types: int, float, string, int list, float list, or string list.

Raises
ValueError if one of the arguments is invalid.

del_hparam

View source

Removes the hyperparameter with key 'name'.

Does nothing if it isn't present.

Args
name Name of the hyperparameter.

from_proto

View source

get

View source

Returns the value of key if it exists, else default.

get_model_structure

View source

override_from_dict

View source

Override existing hyperparameter values, parsing new values from a dictionary.

Args
values_dict Dictionary of name:value pairs.

Returns
The HParams instance.

Raises
KeyError If a hyperparameter in values_dict doesn't exist.
ValueError If values_dict cannot be parsed.

parse

View source

Override existing hyperparameter values, parsing new values from a string.

See parse_values for more detail on the allowed format for values.

Args
values String. Comma separated list of name=value pairs where 'value' must follow the syntax described above.

Returns
The HParams instance.

Raises
ValueError If values cannot be parsed or a hyperparameter in values doesn't exist.

parse_json

View source

Override existing hyperparameter values, parsing new values from a json object.

Args
values_json String containing a json object of name:value pairs.

Returns
The HParams instance.

Raises
KeyError If a hyperparameter in values_json doesn't exist.
ValueError If values_json cannot be parsed.

set_from_map

View source

DEPRECATED. Use override_from_dict. (deprecated)

set_hparam

View source

Set the value of an existing hyperparameter.

This function verifies that the type of the value matches the type of the existing hyperparameter.

Args
name Name of the hyperparameter.
value New value of the hyperparameter.

Raises
KeyError If the hyperparameter doesn't exist.
ValueError If there is a type mismatch.

set_model_structure

View source

to_json

View source

Serializes the hyperparameters into JSON.

Args
indent If a non-negative integer, JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0, or negative, will only insert newlines. None (the default) selects the most compact representation.
separators Optional (item_separator, key_separator) tuple. Default is (', ', ': ').
sort_keys If True, the output dictionaries will be sorted by key.

Returns
A JSON string.

to_proto

View source

Converts a HParams object to a HParamDef protocol buffer.

Args
export_scope Optional string. Name scope to remove.

Returns
A HParamDef protocol buffer.

values

View source

Return the hyperparameter values as a Python dictionary.

Returns
A dictionary with hyperparameter names as keys. The values are the hyperparameter values.

__contains__

View source