This package provides several ops that take care of creating variables that are used internally in a consistent way and provide the building blocks for many common machine learning algorithms.

`tf.contrib.layers.avg_pool2d(*args, **kwargs)`

Adds a 2D average pooling op.

It is assumed that the pooling is done per image but not in batch or channels.

##### Args:

: A`inputs`

`Tensor`

of size [batch_size, height, width, channels].: A list of length 2: [kernel_height, kernel_width] of the pooling kernel over which the op is computed. Can be an int if both values are the same.`kernel_size`

: A list of length 2: [stride_height, stride_width]. Can be an int if both strides are the same. Note that presently both strides must have the same value.`stride`

: The padding method, either 'VALID' or 'SAME'.`padding`

: The collections to which the outputs are added.`outputs_collections`

: Optional scope for name_scope.`scope`

##### Returns:

A `Tensor`

representing the results of the pooling operation.

`tf.contrib.layers.batch_norm(*args, **kwargs)`

Adds a Batch Normalization layer from http://arxiv.org/abs/1502.03167.

"Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift"

Sergey Ioffe, Christian Szegedy

Can be used as a normalizer function for conv2d and fully_connected.

update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS) if update_ops: updates = tf.group(*update_ops) total_loss = control_flow_ops.with_dependencies([updates], total_loss)

One can set update_collections=None to force the updates in place, but that can have speed penalty, specially in distributed settings.

##### Args:

: a tensor with 2 or more dimensions, where the first dimension has`inputs`

`batch_size`

. The normalization is over all but the last dimension.: decay for the moving average.`decay`

: If True, subtract`center`

`beta`

. If False,`beta`

is ignored.: If True, multiply by`scale`

`gamma`

. If False,`gamma`

is not used. When the next layer is linear (also e.g.`nn.relu`

), this can be disabled since the scaling can be done by the next layer.: small float added to variance to avoid dividing by zero.`epsilon`

: activation function, default set to None to skip it and maintain a linear activation.`activation_fn`

: collections to collect the update ops for computation. The updates_ops need to be excuted with the train_op. If None, a control dependency would be added to make sure the updates are computed in place.`updates_collections`

: whether or not the layer is in training mode. In training mode it would accumulate the statistics of the moments into`is_training`

`moving_mean`

and`moving_variance`

using an exponential moving average with the given`decay`

. When it is not in training mode then it would use the values of the`moving_mean`

and the`moving_variance`

.: whether or not the layer and its variables should be reused. To be able to reuse the layer scope must be given.`reuse`

: optional collections for the variables.`variables_collections`

: collections to add the outputs.`outputs_collections`

: If`trainable`

`True`

also add variables to the graph collection`GraphKeys.TRAINABLE_VARIABLES`

(see tf.Variable).: Optional scope for`scope`

`variable_scope`

.

##### Returns:

A `Tensor`

representing the output of the operation.

##### Raises:

: if rank or last dimension of`ValueError`

`inputs`

is undefined.

`tf.contrib.layers.convolution2d(*args, **kwargs)`

Adds a 2D convolution followed by an optional batch_norm layer.

`convolution2d`

creates a variable called `weights`

, representing the
convolutional kernel, that is convolved with the `inputs`

to produce a
`Tensor`

of activations. If a `normalizer_fn`

is provided (such as
`batch_norm`

), it is then applied. Otherwise, if `normalizer_fn`

is
None and a `biases_initializer`

is provided then a `biases`

variable would be
created and added the activations. Finally, if `activation_fn`

is not `None`

,
it is applied to the activations as well.

Performs a'trous convolution with input stride equal to rate if rate is greater than one.

##### Args:

: a 4-D tensor`inputs`

`[batch_size, height, width, channels]`

.: integer, the number of output filters.`num_outputs`

: a list of length 2`kernel_size`

`[kernel_height, kernel_width]`

of of the filters. Can be an int if both values are the same.: a list of length 2`stride`

`[stride_height, stride_width]`

. Can be an int if both strides are the same. Note that presently both strides must have the same value.: one of`padding`

`VALID`

or`SAME`

.: integer. If less than or equal to 1, a standard convolution is used. If greater than 1, than the a'trous convolution is applied and`rate`

`stride`

must be set to 1.: activation function, set to None to skip it and maintain a linear activation.`activation_fn`

: normalization function to use instead of`normalizer_fn`

`biases`

. If`normalizer_fn`

is provided then`biases_initializer`

and`biases_regularizer`

are ignored and`biases`

are not created nor added. default set to None for no normalizer function: normalization function parameters.`normalizer_params`

: An initializer for the weights.`weights_initializer`

: Optional regularizer for the weights.`weights_regularizer`

: An initializer for the biases. If None skip biases.`biases_initializer`

: Optional regularizer for the biases.`biases_regularizer`

: whether or not the layer and its variables should be reused. To be able to reuse the layer scope must be given.`reuse`

: optional list of collections for all the variables or a dictionay containing a different list of collection per variable.`variables_collections`

: collection to add the outputs.`outputs_collections`

: If`trainable`

`True`

also add variables to the graph collection`GraphKeys.TRAINABLE_VARIABLES`

(see tf.Variable).: Optional scope for`scope`

`variable_scope`

.

##### Returns:

a tensor representing the output of the operation.

##### Raises:

: if both 'rate' and`ValueError`

`stride`

are larger than one.

`tf.contrib.layers.convolution2d_in_plane(*args, **kwargs)`

Performs the same in-plane convolution to each channel independently.

This is useful for performing various simple channel-independent convolution operations such as image gradients:

image = tf.constant(..., shape=(16, 240, 320, 3)) vert_gradients = layers.conv2d_in_plane(image, kernel=[1, -1], kernel_size=[2, 1]) horz_gradients = layers.conv2d_in_plane(image, kernel=[1, -1], kernel_size=[1, 2])

##### Args:

: a 4-D tensor with dimensions [batch_size, height, width, channels].`inputs`

: a list of length 2 holding the [kernel_height, kernel_width] of of the pooling. Can be an int if both values are the same.`kernel_size`

: a list of length 2`stride`

`[stride_height, stride_width]`

. Can be an int if both strides are the same. Note that presently both strides must have the same value.: the padding type to use, either 'SAME' or 'VALID'.`padding`

: activation function, set to None to skip it and maintain a linear activation.`activation_fn`

: normalization function to use instead of`normalizer_fn`

`biases`

. If`normalizer_fn`

is provided then`biases_initializer`

and`biases_regularizer`

are ignored and`biases`

are not created nor added. default set to None for no normalizer function: normalization function parameters.`normalizer_params`

: An initializer for the weights.`weights_initializer`

: Optional regularizer for the weights.`weights_regularizer`

: An initializer for the biases. If None skip biases.`biases_initializer`

: Optional regularizer for the biases.`biases_regularizer`

: whether or not the layer and its variables should be reused. To be able to reuse the layer scope must be given.`reuse`

: optional list of collections for all the variables or a dictionay containing a different list of collection per variable.`variables_collections`

: collection to add the outputs.`outputs_collections`

: If`trainable`

`True`

also add variables to the graph collection`GraphKeys.TRAINABLE_VARIABLES`

(see tf.Variable).: Optional scope for`scope`

`variable_scope`

.

##### Returns:

A `Tensor`

representing the output of the operation.

`tf.contrib.layers.convolution2d_transpose(*args, **kwargs)`

Adds a convolution2d_transpose with an optional batch normalization layer.

The function creates a variable called `weights`

, representing the
kernel, that is convolved with the input. If `batch_norm_params`

is `None`

, a
second variable called 'biases' is added to the result of the operation.

##### Args:

: a tensor of size [batch_size, height, width, channels].`inputs`

: integer, the number of output filters.`num_outputs`

: a list of length 2 holding the [kernel_height, kernel_width] of of the filters. Can be an int if both values are the same.`kernel_size`

: a list of length 2: [stride_height, stride_width]. Can be an int if both strides are the same. Note that presently both strides must have the same value.`stride`

: one of 'VALID' or 'SAME'.`padding`

: activation function, set to None to skip it and maintain a linear activation.`activation_fn`

: normalization function to use instead of`normalizer_fn`

`biases`

. If`normalizer_fn`

is provided then`biases_initializer`

and`biases_regularizer`

are ignored and`biases`

are not created nor added. default set to None for no normalizer function: normalization function parameters.`normalizer_params`

: An initializer for the weights.`weights_initializer`

: Optional regularizer for the weights.`weights_regularizer`

: An initializer for the biases. If None skip biases.`biases_initializer`

: Optional regularizer for the biases.`biases_regularizer`

: whether or not the layer and its variables should be reused. To be able to reuse the layer scope must be given.`reuse`

: optional list of collections for all the variables or a dictionay containing a different list of collection per variable.`variables_collections`

: collection to add the outputs.`outputs_collections`

: whether or not the variables should be trainable or not.`trainable`

: Optional scope for variable_scope.`scope`

##### Returns:

a tensor representing the output of the operation.

##### Raises:

: if 'kernel_size' is not a list of length 2.`ValueError`

`tf.contrib.layers.flatten(*args, **kwargs)`

Flattens the input while maintaining the batch_size.

Assumes that the first dimension represents the batch.

##### Args:

: a tensor of size [batch_size, ...].`inputs`

: collection to add the outputs.`outputs_collections`

: Optional scope for name_scope.`scope`

##### Returns:

a flattened tensor with shape [batch_size, k].

##### Raises:

: if inputs.shape is wrong.`ValueError`

`tf.contrib.layers.fully_connected(*args, **kwargs)`

Adds a fully connected layer.

`fully_connected`

creates a variable called `weights`

, representing a fully
connected weight matrix, which is multiplied by the `inputs`

to produce a
`Tensor`

of hidden units. If a `normalizer_fn`

is provided (such as
`batch_norm`

), it is then applied. Otherwise, if `normalizer_fn`

is
None and a `biases_initializer`

is provided then a `biases`

variable would be
created and added the hidden units. Finally, if `activation_fn`

is not `None`

,
it is applied to the hidden units as well.

##### Args:

: A tensor of with at least rank 2 and value for the last dimension, i.e.`inputs`

`[batch_size, depth]`

,`[None, None, None, channels]`

.: Integer or long, the number of output units in the layer.`num_outputs`

: activation function, set to None to skip it and maintain a linear activation.`activation_fn`

: normalization function to use instead of`normalizer_fn`

`biases`

. If`normalizer_fn`

is provided then`biases_initializer`

and`biases_regularizer`

are ignored and`biases`

are not created nor added. default set to None for no normalizer function: normalization function parameters.`normalizer_params`

: An initializer for the weights.`weights_initializer`

: Optional regularizer for the weights.`weights_regularizer`

: An initializer for the biases. If None skip biases.`biases_initializer`

: Optional regularizer for the biases.`biases_regularizer`

: whether or not the layer and its variables should be reused. To be able to reuse the layer scope must be given.`reuse`

: Optional list of collections for all the variables or a dictionary containing a different list of collections per variable.`variables_collections`

: collection to add the outputs.`outputs_collections`

: If`trainable`

`True`

also add variables to the graph collection`GraphKeys.TRAINABLE_VARIABLES`

(see tf.Variable).: Optional scope for variable_scope.`scope`

##### Returns:

the tensor variable representing the result of the series of operations.

##### Raises:

: if x has rank less than 2 or if its last dimension is not set.`ValueError`

`tf.contrib.layers.layer_norm(*args, **kwargs)`

Adds a Layer Normalization layer from https://arxiv.org/abs/1607.06450.

"Layer Normalization"

Jimmy Lei Ba, Jamie Ryan Kiros, Geoffrey E. Hinton

Can be used as a normalizer function for conv2d and fully_connected.

##### Args:

: a tensor with 2 or more dimensions. The normalization occurs over all but the first dimension.`inputs`

: If True, subtract`center`

`beta`

. If False,`beta`

is ignored.: If True, multiply by`scale`

`gamma`

. If False,`gamma`

is not used. When the next layer is linear (also e.g.`nn.relu`

), this can be disabled since the scaling can be done by the next layer.: activation function, default set to None to skip it and maintain a linear activation.`activation_fn`

: whether or not the layer and its variables should be reused. To be able to reuse the layer scope must be given.`reuse`

: optional collections for the variables.`variables_collections`

: collections to add the outputs.`outputs_collections`

: If`trainable`

`True`

also add variables to the graph collection`GraphKeys.TRAINABLE_VARIABLES`

(see tf.Variable).: Optional scope for`scope`

`variable_op_scope`

.

##### Returns:

A `Tensor`

representing the output of the operation.

##### Raises:

: if rank or last dimension of`ValueError`

`inputs`

is undefined.

`tf.contrib.layers.max_pool2d(*args, **kwargs)`

Adds a 2D Max Pooling op.

It is assumed that the pooling is done per image but not in batch or channels.

##### Args:

: A`inputs`

`Tensor`

of size [batch_size, height, width, channels].: A list of length 2: [kernel_height, kernel_width] of the pooling kernel over which the op is computed. Can be an int if both values are the same.`kernel_size`

: A list of length 2: [stride_height, stride_width]. Can be an int if both strides are the same. Note that presently both strides must have the same value.`stride`

: The padding method, either 'VALID' or 'SAME'.`padding`

: The collections to which the outputs are added.`outputs_collections`

: Optional scope for name_scope.`scope`

##### Returns:

A `Tensor`

representing the results of the pooling operation.

##### Raises:

: If 'kernel_size' is not a 2-D list`ValueError`

`tf.contrib.layers.one_hot_encoding(*args, **kwargs)`

Transform numeric labels into onehot_labels using tf.one_hot.

##### Args:

: [batch_size] target labels.`labels`

: total number of classes.`num_classes`

: A scalar defining the on-value.`on_value`

: A scalar defining the off-value.`off_value`

: collection to add the outputs.`outputs_collections`

: Optional scope for name_scope.`scope`

##### Returns:

one hot encoding of the labels.

`tf.contrib.layers.repeat(inputs, repetitions, layer, *args, **kwargs)`

Applies the same layer with the same arguments repeatedly.

```
y = repeat(x, 3, conv2d, 64, [3, 3], scope='conv1')
# It is equivalent to:
x = conv2d(x, 64, [3, 3], scope='conv1/conv1_1')
x = conv2d(x, 64, [3, 3], scope='conv1/conv1_2')
y = conv2d(x, 64, [3, 3], scope='conv1/conv1_3')
```

If the `scope`

argument is not given in `kwargs`

, it is set to
`layer.__name__`

, or `layer.func.__name__`

(for `functools.partial`

objects). If neither `__name__`

nor `func.__name__`

is available, the
layers are called with `scope='stack'`

.

##### Args:

: A`inputs`

`Tensor`

suitable for layer.: Int, number of repetitions.`repetitions`

: A layer with arguments`layer`

`(inputs, *args, **kwargs)`

: Extra args for the layer.`*args`

: Extra kwargs for the layer.`**kwargs`

##### Returns:

a tensor result of applying the layer, repetitions times.

##### Raises:

: if the op is unknown or wrong.`ValueError`

`tf.contrib.layers.safe_embedding_lookup_sparse(embedding_weights, sparse_ids, sparse_weights=None, combiner=None, default_id=None, name=None, partition_strategy='div')`

Lookup embedding results, accounting for invalid IDs and empty features.

The partitioned embedding in `embedding_weights`

must all be the same shape
except for the first dimension. The first dimension is allowed to vary as the
vocabulary size is not necessarily a multiple of `P`

.

Invalid IDs (< 0) are pruned from input IDs and weights, as well as any IDs
with non-positive weight. For an entry with no features, the embedding vector
for `default_id`

is returned, or the 0-vector if `default_id`

is not supplied.

The ids and weights may be multi-dimensional. Embeddings are always aggregated along the last dimension.

##### Args:

: A list of`embedding_weights`

`P`

float tensors or values representing partitioned embedding tensors. The total unpartitioned shape should be`[e_0, e_1, ..., e_m]`

, where`e_0`

represents the vocab size and`e_1, ..., e_m`

are the embedding dimensions.:`sparse_ids`

`SparseTensor`

of shape`[d_0, d_1, ..., d_n]`

containing the ids.`d_0`

is typically batch size.:`sparse_weights`

`SparseTensor`

of same shape as`sparse_ids`

, containing float weights corresponding to`sparse_ids`

, or`None`

if all weights are be assumed to be 1.0.: A string specifying how to combine embedding results for each entry. Currently "mean", "sqrtn" and "sum" are supported, with "mean" the default.`combiner`

: The id to use for an entry with no features.`default_id`

: A name for this operation (optional).`name`

: A string specifying the partitioning strategy. Currently`partition_strategy`

`"div"`

and`"mod"`

are supported. Default is`"div"`

.

##### Returns:

Dense tensor of shape `[d_0, d_1, ..., d_{n-1}, e_1, ..., e_m]`

.

##### Raises:

: if`ValueError`

`embedding_weights`

is empty.

`tf.contrib.layers.separable_convolution2d(*args, **kwargs)`

Adds a depth-separable 2D convolution with optional batch_norm layer.

This op first performs a depthwise convolution that acts separately on
channels, creating a variable called `depthwise_weights`

. If `num_outputs`

is not None, it adds a pointwise convolution that mixes channels, creating a
variable called `pointwise_weights`

. Then, if `batch_norm_params`

is None,
it adds bias to the result, creating a variable called 'biases', otherwise
it adds a batch normalization layer. It finally applies an activation function
to produce the end result.

##### Args:

: a tensor of size [batch_size, height, width, channels].`inputs`

: the number of pointwise convolution output filters. If is None, then we skip the pointwise convolution stage.`num_outputs`

: a list of length 2: [kernel_height, kernel_width] of of the filters. Can be an int if both values are the same.`kernel_size`

: the number of depthwise convolution output channels for each input channel. The total number of depthwise convolution output channels will be equal to`depth_multiplier`

`num_filters_in * depth_multiplier`

.: a list of length 2: [stride_height, stride_width], specifying the depthwise convolution stride. Can be an int if both strides are the same.`stride`

: one of 'VALID' or 'SAME'.`padding`

: activation function, set to None to skip it and maintain a linear activation.`activation_fn`

: normalization function to use instead of`normalizer_fn`

`biases`

. If`normalizer_fn`

is provided then`biases_initializer`

and`biases_regularizer`

are ignored and`biases`

are not created nor added. default set to None for no normalizer function: normalization function parameters.`normalizer_params`

: An initializer for the weights.`weights_initializer`

: Optional regularizer for the weights.`weights_regularizer`

: An initializer for the biases. If None skip biases.`biases_initializer`

: Optional regularizer for the biases.`biases_regularizer`

: whether or not the layer and its variables should be reused. To be able to reuse the layer scope must be given.`reuse`

: optional list of collections for all the variables or a dictionay containing a different list of collection per variable.`variables_collections`

: collection to add the outputs.`outputs_collections`

: whether or not the variables should be trainable or not.`trainable`

: Optional scope for variable_scope.`scope`

##### Returns:

A `Tensor`

representing the output of the operation.

`tf.contrib.layers.stack(inputs, layer, stack_args, **kwargs)`

Builds a stack of layers by applying layer repeatedly using stack_args.

`stack`

allows you to repeatedly apply the same operation with different
arguments `stack_args[i]`

. For each application of the layer, `stack`

creates
a new scope appended with an increasing number. For example:

```
y = stack(x, fully_connected, [32, 64, 128], scope='fc')
# It is equivalent to:
x = fully_connected(x, 32, scope='fc/fc_1')
x = fully_connected(x, 64, scope='fc/fc_2')
y = fully_connected(x, 128, scope='fc/fc_3')
```

If the `scope`

argument is not given in `kwargs`

, it is set to
`layer.__name__`

, or `layer.func.__name__`

(for `functools.partial`

objects). If neither `__name__`

nor `func.__name__`

is available, the
layers are called with `scope='stack'`

.

##### Args:

: A`inputs`

`Tensor`

suitable for layer.: A layer with arguments`layer`

`(inputs, *args, **kwargs)`

: A list/tuple of parameters for each call of layer.`stack_args`

: Extra kwargs for the layer.`**kwargs`

##### Returns:

a `Tensor`

result of applying the stacked layers.

##### Raises:

: if the op is unknown or wrong.`ValueError`

`tf.contrib.layers.unit_norm(*args, **kwargs)`

Normalizes the given input across the specified dimension to unit length.

Note that the rank of `input`

must be known.

##### Args:

: A`inputs`

`Tensor`

of arbitrary size.: The dimension along which the input is normalized.`dim`

: A small value to add to the inputs to avoid dividing by zero.`epsilon`

: Optional scope for variable_scope.`scope`

##### Returns:

The normalized `Tensor`

.

##### Raises:

: If dim is smaller than the number of dimensions in 'inputs'.`ValueError`

Aliases for fully_connected which set a default activation function are
available: `relu`

, `relu6`

and `linear`

.