Join us at TensorFlow World, Oct 28-31. Use code TF20 for 20% off select passes.

# tf.contrib.constrained_optimization.MultiplicativeSwapRegretOptimizer

## Class `MultiplicativeSwapRegretOptimizer`

A `ConstrainedOptimizer` based on swap-regret minimization.

This `ConstrainedOptimizer` uses the given `tf.compat.v1.train.Optimizer`s to jointly minimize over the model parameters, and maximize over constraint/objective weight matrix (the analogue of Lagrange multipliers), with the latter maximization using multiplicative updates and an algorithm that minimizes swap regret.

For more specifics, please refer to:

Cotter, Jiang and Sridharan. "Two-Player Games for Efficient Non-Convex Constrained Optimization". https://arxiv.org/abs/1804.06500

The formulation used by this optimizer can be found in Definition 2, and is discussed in Section 4. It is most similar to Algorithm 2 in Section 4, with the difference being that it uses `tf.compat.v1.train.Optimizer`s, instead of SGD, for the "inner" updates.

## `__init__`

View source

``````__init__(
optimizer,
constraint_optimizer=None,
)
``````

Constructs a new `MultiplicativeSwapRegretOptimizer`.

#### Args:

• `optimizer`: tf.compat.v1.train.Optimizer, used to optimize the objective and proxy_constraints portion of ConstrainedMinimizationProblem. If constraint_optimizer is not provided, this will also be used to optimize the Lagrange multiplier analogues.
• `constraint_optimizer`: optional tf.compat.v1.train.Optimizer, used to optimize the Lagrange multiplier analogues.
• `minimum_multiplier_radius`: float, each element of the matrix will be lower bounded by `minimum_multiplier_radius` divided by one plus the number of constraints.
• `initial_multiplier_radius`: float, the initial value of each element of the matrix associated with a constraint (i.e. excluding those elements associated with the objective) will be `initial_multiplier_radius` divided by one plus the number of constraints. Defaults to the value of `minimum_multiplier_radius`.

#### Returns:

A new `MultiplicativeSwapRegretOptimizer`.

#### Raises:

• `ValueError`: If the two radius parameters are inconsistent.

## Properties

### `constraint_optimizer`

Returns the `tf.compat.v1.train.Optimizer` used for the matrix.

### `optimizer`

Returns the `tf.compat.v1.train.Optimizer` used for optimization.

## Methods

### `minimize`

View source

``````minimize(
minimization_problem,
unconstrained_steps=None,
global_step=None,
var_list=None,
aggregation_method=None,
name=None,
)
``````

Returns an `Operation` for minimizing the constrained problem.

This method combines the functionality of `minimize_unconstrained` and `minimize_constrained`. If global_step < unconstrained_steps, it will perform an unconstrained update, and if global_step >= unconstrained_steps, it will perform a constrained update.

The reason for this functionality is that it may be best to initialize the constrained optimizer with an approximate optimum of the unconstrained problem.

#### Returns:

`Operation`, the train_op.

#### Raises:

• `ValueError`: If unconstrained_steps is provided, but global_step is not.

### `minimize_constrained`

View source

``````minimize_constrained(
minimization_problem,
global_step=None,
var_list=None,
aggregation_method=None,
name=None,
)
``````

Returns an `Operation` for minimizing the constrained problem.

Unlike `minimize_unconstrained`, this function attempts to find a solution that minimizes the `objective` portion of the minimization problem while satisfying the `constraints` portion.

#### Returns:

`Operation`, the train_op.

### `minimize_unconstrained`

View source

``````minimize_unconstrained(
minimization_problem,
global_step=None,
var_list=None,
aggregation_method=None,
name=None,
Returns an `Operation` for minimizing the unconstrained problem.
Unlike `minimize_constrained`, this function ignores the `constraints` (and `proxy_constraints`) portion of the minimization problem entirely, and only minimizes `objective`.
`Operation`, the train_op.