|---|
| [ View source on GitHub](https://github.com/tensorflow/probability/blob/v0.25.0/tensorflow_probability/python/optimizer/proximal_hessian_sparse.py#L461-L570) |
Minimize using Hessian-informed proximal gradient descent.
tfp.optimizer.proximal_hessian_sparse_minimize(
grad_and_hessian_loss_fn,
x_start,
tolerance,
l1_regularizer,
l2_regularizer=None,
maximum_iterations=1,
maximum_full_sweeps_per_iteration=1,
learning_rate=None,
name=None
)
This function solves the regularized minimization problem
argmin{ Loss(x)
+ l1_regularizer * ||x||_1
+ l2_regularizer * ||x||_2**2
: x in R^n }
where `Loss` is a convex C\^2 function (typically, `Loss` is the negative log
likelihood of a model and `x` is a vector of model coefficients). The `Loss`
function does not need to be supplied directly, but this optimizer does need a
way to compute the gradient and Hessian of the Loss function at a given value
of `x`. The gradient and Hessian are often computationally expensive, and
this optimizer calls them relatively few times compared with other algorithms.
| ## Args ||
|---|---|
| `grad_and_hessian_loss_fn` | callable that takes as input a (batch of) `Tensor` of the same shape and dtype as `x_start` and returns the triple `(gradient_unregularized_loss, hessian_unregularized_loss_outer, hessian_unregularized_loss_middle)` as defined in the argument spec of `minimize_one_step`. |
| `x_start` | (Batch of) vector-shaped, `float` `Tensor` representing the initial value of the argument to the `Loss` function. |
| `tolerance` | scalar, `float` `Tensor` representing the tolerance for each optimization step; see the `tolerance` argument of `minimize_one_step`. |
| `l1_regularizer` | scalar, `float` `Tensor` representing the weight of the L1 regularization term (see equation above). |
| `l2_regularizer` | scalar, `float` `Tensor` representing the weight of the L2 regularization term (see equation above). Default value: `None` (i.e., no L2 regularization). |
| `maximum_iterations` | Python integer specifying the maximum number of iterations of the outer loop of the optimizer. After this many iterations of the outer loop, the algorithm will terminate even if the return value `optimal_x` has not converged. Default value: `1`. |
| `maximum_full_sweeps_per_iteration` | Python integer specifying the maximum number of sweeps allowed in each iteration of the outer loop of the optimizer. Passed as the `maximum_full_sweeps` argument to `minimize_one_step`. Default value: `1`. |
| `learning_rate` | scalar, `float` `Tensor` representing a multiplicative factor used to dampen the proximal gradient descent steps. Default value: `None` (i.e., factor is conceptually `1`). |
| `name` | Python string representing the name of the TensorFlow operation. The default name is `"minimize"`. |
| ## Returns ||
|---|---|
| `x` | `Tensor` of the same shape and dtype as `x_start`, representing the (batches of) computed values of `x` which minimizes `Loss(x)`. |
| `is_converged` | scalar, `bool` `Tensor` indicating whether the minimization procedure converged within the specified number of iterations across all batches. Here convergence means that an iteration of the inner loop (`minimize_one_step`) returns `True` for its `is_converged` output value. |
| `iter` | scalar, `int` `Tensor` indicating the actual number of iterations of the outer loop of the optimizer completed (i.e., number of calls to `minimize_one_step` before achieving convergence). |
#### References
\[1\]: Jerome Friedman, Trevor Hastie and Rob Tibshirani. Regularization Paths
for Generalized Linear Models via Coordinate Descent. *Journal of
Statistical Software* , 33(1), 2010.
\[2\]: Guo-Xun Yuan, Chia-Hua Ho and Chih-Jen Lin. An Improved GLMNET for
L1-regularized Logistic Regression. *Journal of Machine Learning
Research* , 13, 2012.