# tf.contrib.layers.group_norm

<br />

|---|
| [![](https://www.tensorflow.org/images/GitHub-Mark-32px.png) View source on GitHub](https://github.com/tensorflow/tensorflow/blob/v1.15.0/tensorflow/contrib/layers/python/layers/normalization.py#L172-L391) |

Functional interface for the group normalization layer.

    tf.contrib.layers.group_norm(
        inputs, groups=32, channels_axis=-1, reduction_axes=(-3, -2), center=True,
        scale=True, epsilon=1e-06, activation_fn=None, param_initializers=None,
        reuse=None, variables_collections=None, outputs_collections=None,
        trainable=True, scope=None, mean_close_to_zero=False
    )

Reference: <https://arxiv.org/abs/1803.08494>

"Group Normalization", Yuxin Wu, Kaiming He

<br />

<br />

<br />

| ## Args ||
|---|---|
| `inputs` | A Tensor with at least 2 dimensions one which is channels. All shape dimensions except for batch must be fully defined. |
| `groups` | Integer. Divide the channels into this number of groups over which normalization statistics are computed. This number must be commensurate with the number of channels in `inputs`. |
| `channels_axis` | An integer. Specifies index of channels axis which will be broken into `groups`, each of which whose statistics will be computed across. Must be mutually exclusive with `reduction_axes`. Preferred usage is to specify negative integers to be agnostic as to whether a batch dimension is included. |
| `reduction_axes` | Tuple of integers. Specifies dimensions over which statistics will be accumulated. Must be mutually exclusive with `channels_axis`. Statistics will not be accumulated across axes not specified in `reduction_axes` nor `channel_axis`. Preferred usage is to specify negative integers to be agnostic to whether a batch dimension is included. <br /> Some sample usage cases: NHWC format: channels_axis=-1, reduction_axes=\[-3, -2\] NCHW format: channels_axis=-3, reduction_axes=\[-2, -1\] |
| `center` | If True, add offset of `beta` to normalized tensor. If False, `beta` is ignored. |
| `scale` | If True, multiply by `gamma`. If False, `gamma` is not used. When the next layer is linear (also e.g. [`nn.relu`](https://www.tensorflow.org/api_docs/python/tf/nn/relu)), this can be disabled since the scaling can be done by the next layer. |
| `epsilon` | Small float added to variance to avoid dividing by zero. |
| `activation_fn` | Activation function, default set to None to skip it and maintain a linear activation. |
| `param_initializers` | Optional initializers for beta, gamma, moving mean and moving variance. |
| `reuse` | Whether or not the layer and its variables should be reused. To be able to reuse the layer scope must be given. |
| `variables_collections` | Optional collections for the variables. |
| `outputs_collections` | Collections to add the outputs. |
| `trainable` | If `True` also add variables to the graph collection `GraphKeys.TRAINABLE_VARIABLES` (see [`tf.Variable`](https://www.tensorflow.org/versions/r1.15/api_docs/python/tf/Variable)). |
| `scope` | Optional scope for `variable_scope`. |
| `mean_close_to_zero` | The mean of `input` before ReLU will be close to zero when batch size \>= 4k for Resnet-50 on TPU. If `True`, use [`nn.sufficient_statistics`](https://www.tensorflow.org/api_docs/python/tf/nn/sufficient_statistics) and [`nn.normalize_moments`](https://www.tensorflow.org/api_docs/python/tf/nn/normalize_moments) to calculate the variance. This is the same behavior as `fused` equals `True` in batch normalization. If `False`, use [`nn.moments`](https://www.tensorflow.org/api_docs/python/tf/nn/moments) to calculate the variance. When `mean` is close to zero, like 1e-4, use `mean` to calculate the variance may have poor result due to repeated roundoff error and denormalization in `mean`. When `mean` is large, like 1e2, sum(`input`\^2) is so large that only the high-order digits of the elements are being accumulated. Thus, use sum(`input` - `mean`)\^2/n to calculate the variance has better accuracy compared to (sum(`input`\^2)/n - `mean`\^2) when `mean` is large. |

<br />

<br />

<br />

<br />

| ## Returns ||
|---|---|
| A `Tensor` representing the output of the operation. ||

<br />

<br />

<br />

<br />

| ## Raises ||
|---|---|
| `ValueError` | If the rank of `inputs` is undefined. |
| `ValueError` | If rank or channels dimension of `inputs` is undefined. |
| `ValueError` | If number of groups is not commensurate with number of channels. |
| `ValueError` | If reduction_axes or channels_axis are out of bounds. |
| `ValueError` | If reduction_axes are not mutually exclusive with channels_axis. |

<br />