Join us at TensorFlow World, Oct 28-31. Use code TF20 for 20% off select passes. Register now

nsl.lib.virtual_adv_regularizer

View source on GitHub

API to calculate virtual adversarial loss for given input.

nsl.lib.virtual_adv_regularizer(
    input_layer,
    embedding_fn,
    virtual_adv_config,
    embedding=None
)

Virtual adversarial loss is defined as the distance between the embedding of input and that of slightly perturbed input. Optimizing this loss helps smoothen models locally.

Reference paper: https://arxiv.org/pdf/1704.03976.pdf

Args:

  • input_layer: a dense tensor for input features, with batch_size as the 1st dimension.
  • embedding_fn: a unary function that takes the input layer to compute and return its embedding.
  • virtual_adv_config: a VirtualAdvConfig object.
  • embedding: (optional) a dense tensor for embedding of the input_layer. If not provided, it will be calculated by embedding_fn(input_layer).

Returns:

  • virtual_adv_loss: a scalar (float32) for virtural adversarial loss.