Missed TensorFlow World? Check out the recap. Learn more

nsl.lib.virtual_adv_regularizer

View source on GitHub

Calculates virtual adversarial loss for the given input.

nsl.lib.virtual_adv_regularizer(
    input_layer,
    embedding_fn,
    virtual_adv_config,
    embedding=None
)

Virtual adversarial loss is defined as the distance between the embedding of the input and that of a slightly perturbed input. Optimizing this loss helps smooth models locally.

Reference paper: https://arxiv.org/pdf/1704.03976.pdf

Args:

  • input_layer: a dense tensor for input features whose first dimension is the training batch size.
  • embedding_fn: a unary function that computes the embedding for the given input_layer input.
  • virtual_adv_config: an nsl.configs.VirtualAdvConfig object that specifies parameters for generating adversarial examples and computing the the adversarial loss.
  • embedding: (optional) a dense tensor representing the embedding of input_layer. If not provided, it will be calculated as embedding_fn(input_layer).

Returns:

  • virtual_adv_loss: a float32 denoting the virtural adversarial loss.