View source on GitHub |
Structured interpolation to approximate a large matrix.
tfp.experimental.linalg.LinearOperatorInterpolatedPSDKernel(
kernel,
bounds_min,
bounds_max,
num_interpolation_points,
x1,
x2=None,
interp_fn=tfp.math.batch_interp_regular_nd_grid
,
diag_shift=None,
is_non_singular=None,
is_self_adjoint=None,
is_positive_definite=None,
is_square=None,
name=None
)
This implements a component of the Structured Kernel Interpolation [1]
algorithm. We approximate the pairwise kernel
values for two inputs x1
,
x2
with the following product:
k(x1, x2) = r(x1) @ k(u, u) @ r(x2)^T
where u
is a set of points regularly spaced on grid and r
is an
interpolation matrix. In short, instead of evaluating kernel
on x1
and
x2
, we evaluate it on u
, which is chosen to contain fewer points than x1
and x2
and the interpolate using the interpolation matrix.
This construction lets us compute matrix products efficiently. If x1
/x2
are of shape O(n)
and u
is of shape O(m)
, this reduces the compute cost
and memory to O(n + m^2)
from O(n^2)
.
In practice, the interpolation matrix is implicitly defined using interp_fn
,
of which tfp.math.batch_interp_regular_nd_grid
linear interpolation is a
prototypical example. When x1 == x2
we can relatively cheaply compute the
diagonal of that matrix exactly, to preserve positive-semi-definiteness.
Since we rely on a dense grid u
, this method works best when the the kernel
inputs are low dimensional (2 or 3).
References
[1]: Wilson, A. G., & Nickisch, H. Kernel Interpolation for Scalable Structured Gaussian Processes (KISS-GP). 2005. arXiv. http://arxiv.org/abs/1503.01057
Attributes | |
---|---|
H
|
Returns the adjoint of the current LinearOperator .
Given |
batch_shape
|
TensorShape of batch dimensions of this LinearOperator .
If this operator acts like the batch matrix |
bounds_max
|
|
bounds_min
|
|
diag_shift
|
|
domain_dimension
|
Dimension (in the sense of vector spaces) of the domain of this operator.
If this operator acts like the batch matrix |
dtype
|
The DType of Tensor s handled by this LinearOperator .
|
graph_parents
|
List of graph dependencies of this LinearOperator . (deprecated)
|
is_non_singular
|
|
is_positive_definite
|
|
is_self_adjoint
|
|
is_square
|
Return True/False depending on if this operator is square.
|
kernel
|
|
name
|
Name prepended to all ops created by this LinearOperator .
|
name_scope
|
Returns a tf.name_scope instance for this class.
|
non_trainable_variables
|
Sequence of non-trainable variables owned by this module and its submodules. |
num_interpolation_points
|
|
parameters
|
Dictionary of parameters used to instantiate this LinearOperator .
|
range_dimension
|
Dimension (in the sense of vector spaces) of the range of this operator.
If this operator acts like the batch matrix |
shape
|
TensorShape of this LinearOperator .
If this operator acts like the batch matrix |
submodules
|
Sequence of all sub-modules.
Submodules are modules which are properties of this module, or found as properties of modules which are properties of this module (and so on).
|
tensor_rank
|
Rank (in the sense of tensors) of matrix corresponding to this operator.
If this operator acts like the batch matrix |
trainable_variables
|
Sequence of trainable variables owned by this module and its submodules. |
variables
|
Sequence of variables owned by this module and its submodules. |
x1
|
|
x2
|
Methods
add_to_tensor
add_to_tensor(
x, name='add_to_tensor'
)
Add matrix represented by this operator to x
. Equivalent to A + x
.
Args | |
---|---|
x
|
Tensor with same dtype and shape broadcastable to self.shape .
|
name
|
A name to give this Op .
|
Returns | |
---|---|
A Tensor with broadcast shape and same dtype as self .
|
adjoint
adjoint(
name: str = 'adjoint'
) -> 'LinearOperator'
Returns the adjoint of the current LinearOperator
.
Given A
representing this LinearOperator
, return A*
.
Note that calling self.adjoint()
and self.H
are equivalent.
Args | |
---|---|
name
|
A name for this Op .
|
Returns | |
---|---|
LinearOperator which represents the adjoint of this LinearOperator .
|
assert_non_singular
assert_non_singular(
name='assert_non_singular'
)
Returns an Op
that asserts this operator is non singular.
This operator is considered non-singular if
ConditionNumber < max{100, range_dimension, domain_dimension} * eps,
eps := np.finfo(self.dtype.as_numpy_dtype).eps
Args | |
---|---|
name
|
A string name to prepend to created ops. |
Returns | |
---|---|
An Assert Op , that, when run, will raise an InvalidArgumentError if
the operator is singular.
|
assert_positive_definite
assert_positive_definite(
name='assert_positive_definite'
)
Returns an Op
that asserts this operator is positive definite.
Here, positive definite means that the quadratic form x^H A x
has positive
real part for all nonzero x
. Note that we do not require the operator to
be self-adjoint to be positive definite.
Args | |
---|---|
name
|
A name to give this Op .
|
Returns | |
---|---|
An Assert Op , that, when run, will raise an InvalidArgumentError if
the operator is not positive definite.
|
assert_self_adjoint
assert_self_adjoint(
name='assert_self_adjoint'
)
Returns an Op
that asserts this operator is self-adjoint.
Here we check that this operator is exactly equal to its hermitian transpose.
Args | |
---|---|
name
|
A string name to prepend to created ops. |
Returns | |
---|---|
An Assert Op , that, when run, will raise an InvalidArgumentError if
the operator is not self-adjoint.
|
batch_shape_tensor
batch_shape_tensor(
name='batch_shape_tensor'
)
Shape of batch dimensions of this operator, determined at runtime.
If this operator acts like the batch matrix A
with
A.shape = [B1,...,Bb, M, N]
, then this returns a Tensor
holding
[B1,...,Bb]
.
Args | |
---|---|
name
|
A name for this Op .
|
Returns | |
---|---|
int32 Tensor
|
cholesky
cholesky(
name: str = 'cholesky'
) -> 'LinearOperator'
Returns a Cholesky factor as a LinearOperator
.
Given A
representing this LinearOperator
, if A
is positive definite
self-adjoint, return L
, where A = L L^T
, i.e. the cholesky
decomposition.
Args | |
---|---|
name
|
A name for this Op .
|
Returns | |
---|---|
LinearOperator which represents the lower triangular matrix
in the Cholesky decomposition.
|
Raises | |
---|---|
ValueError
|
When the LinearOperator is not hinted to be positive
definite and self adjoint.
|
cond
cond(
name='cond'
)
Returns the condition number of this linear operator.
Args | |
---|---|
name
|
A name for this Op .
|
Returns | |
---|---|
Shape [B1,...,Bb] Tensor of same dtype as self .
|
determinant
determinant(
name='det'
)
Determinant for every batch member.
Args | |
---|---|
name
|
A name for this Op .
|
Returns | |
---|---|
Tensor with shape self.batch_shape and same dtype as self .
|
Raises | |
---|---|
NotImplementedError
|
If self.is_square is False .
|
diag_part
diag_part(
name='diag_part'
)
Efficiently get the [batch] diagonal part of this operator.
If this operator has shape [B1,...,Bb, M, N]
, this returns a
Tensor
diagonal
, of shape [B1,...,Bb, min(M, N)]
, where
diagonal[b1,...,bb, i] = self.to_dense()[b1,...,bb, i, i]
.
my_operator = LinearOperatorDiag([1., 2.])
# Efficiently get the diagonal
my_operator.diag_part()
==> [1., 2.]
# Equivalent, but inefficient method
tf.linalg.diag_part(my_operator.to_dense())
==> [1., 2.]
Args | |
---|---|
name
|
A name for this Op .
|
Returns | |
---|---|
diag_part
|
A Tensor of same dtype as self.
|
domain_dimension_tensor
domain_dimension_tensor(
name='domain_dimension_tensor'
)
Dimension (in the sense of vector spaces) of the domain of this operator.
Determined at runtime.
If this operator acts like the batch matrix A
with
A.shape = [B1,...,Bb, M, N]
, then this returns N
.
Args | |
---|---|
name
|
A name for this Op .
|
Returns | |
---|---|
int32 Tensor
|
eigvals
eigvals(
name='eigvals'
)
Returns the eigenvalues of this linear operator.
If the operator is marked as self-adjoint (via is_self_adjoint
)
this computation can be more efficient.
Args | |
---|---|
name
|
A name for this Op .
|
Returns | |
---|---|
Shape [B1,...,Bb, N] Tensor of same dtype as self .
|
inverse
inverse(
name: str = 'inverse'
) -> 'LinearOperator'
Returns the Inverse of this LinearOperator
.
Given A
representing this LinearOperator
, return a LinearOperator
representing A^-1
.
Args | |
---|---|
name
|
A name scope to use for ops added by this method. |
Returns | |
---|---|
LinearOperator representing inverse of this matrix.
|
Raises | |
---|---|
ValueError
|
When the LinearOperator is not hinted to be non_singular .
|
log_abs_determinant
log_abs_determinant(
name='log_abs_det'
)
Log absolute value of determinant for every batch member.
Args | |
---|---|
name
|
A name for this Op .
|
Returns | |
---|---|
Tensor with shape self.batch_shape and same dtype as self .
|
Raises | |
---|---|
NotImplementedError
|
If self.is_square is False .
|
matmul
matmul(
x, adjoint=False, adjoint_arg=False, name='matmul'
)
Transform [batch] matrix x
with left multiplication: x --> Ax
.
# Make an operator acting like batch matrix A. Assume A.shape = [..., M, N]
operator = LinearOperator(...)
operator.shape = [..., M, N]
X = ... # shape [..., N, R], batch matrix, R > 0.
Y = operator.matmul(X)
Y.shape
==> [..., M, R]
Y[..., :, r] = sum_j A[..., :, j] X[j, r]
Args | |
---|---|
x
|
LinearOperator or Tensor with compatible shape and same dtype as
self . See class docstring for definition of compatibility.
|
adjoint
|
Python bool . If True , left multiply by the adjoint: A^H x .
|
adjoint_arg
|
Python bool . If True , compute A x^H where x^H is
the hermitian transpose (transposition and complex conjugation).
|
name
|
A name for this Op .
|
Returns | |
---|---|
A LinearOperator or Tensor with shape [..., M, R] and same dtype
as self .
|
matvec
matvec(
x, adjoint=False, name='matvec'
)
Transform [batch] vector x
with left multiplication: x --> Ax
.
# Make an operator acting like batch matrix A. Assume A.shape = [..., M, N]
operator = LinearOperator(...)
X = ... # shape [..., N], batch vector
Y = operator.matvec(X)
Y.shape
==> [..., M]
Y[..., :] = sum_j A[..., :, j] X[..., j]
Args | |
---|---|
x
|
Tensor with compatible shape and same dtype as self .
x is treated as a [batch] vector meaning for every set of leading
dimensions, the last dimension defines a vector.
See class docstring for definition of compatibility.
|
adjoint
|
Python bool . If True , left multiply by the adjoint: A^H x .
|
name
|
A name for this Op .
|
Returns | |
---|---|
A Tensor with shape [..., M] and same dtype as self .
|
range_dimension_tensor
range_dimension_tensor(
name='range_dimension_tensor'
)
Dimension (in the sense of vector spaces) of the range of this operator.
Determined at runtime.
If this operator acts like the batch matrix A
with
A.shape = [B1,...,Bb, M, N]
, then this returns M
.
Args | |
---|---|
name
|
A name for this Op .
|
Returns | |
---|---|
int32 Tensor
|
row
row(
index
)
Gets a row from the dense operator.
Args | |
---|---|
index
|
The index (indices) of the row[s] to get, may be scalar or up to batch shape. |
Returns | |
---|---|
rows
|
Row[s] of the matrix, with shape (...batch_shape..., num_cols) .
Effectively the same as operator.to_dense()[..., index, :] for a
scalar index , analogous to gather for non-scalar.
|
shape_tensor
shape_tensor(
name='shape_tensor'
)
Shape of this LinearOperator
, determined at runtime.
If this operator acts like the batch matrix A
with
A.shape = [B1,...,Bb, M, N]
, then this returns a Tensor
holding
[B1,...,Bb, M, N]
, equivalent to tf.shape(A)
.
Args | |
---|---|
name
|
A name for this Op .
|
Returns | |
---|---|
int32 Tensor
|
solve
solve(
rhs, adjoint=False, adjoint_arg=False, name='solve'
)
Solve (exact or approx) R
(batch) systems of equations: A X = rhs
.
The returned Tensor
will be close to an exact solution if A
is well
conditioned. Otherwise closeness will vary. See class docstring for details.
Examples:
# Make an operator acting like batch matrix A. Assume A.shape = [..., M, N]
operator = LinearOperator(...)
operator.shape = [..., M, N]
# Solve R > 0 linear systems for every member of the batch.
RHS = ... # shape [..., M, R]
X = operator.solve(RHS)
# X[..., :, r] is the solution to the r'th linear system
# sum_j A[..., :, j] X[..., j, r] = RHS[..., :, r]
operator.matmul(X)
==> RHS
Args | |
---|---|
rhs
|
Tensor with same dtype as this operator and compatible shape.
rhs is treated like a [batch] matrix meaning for every set of leading
dimensions, the last two dimensions defines a matrix.
See class docstring for definition of compatibility.
|
adjoint
|
Python bool . If True , solve the system involving the adjoint
of this LinearOperator : A^H X = rhs .
|
adjoint_arg
|
Python bool . If True , solve A X = rhs^H where rhs^H
is the hermitian transpose (transposition and complex conjugation).
|
name
|
A name scope to use for ops added by this method. |
Returns | |
---|---|
Tensor with shape [...,N, R] and same dtype as rhs .
|
Raises | |
---|---|
NotImplementedError
|
If self.is_non_singular or is_square is False.
|
solvevec
solvevec(
rhs, adjoint=False, name='solve'
)
Solve single equation with best effort: A X = rhs
.
The returned Tensor
will be close to an exact solution if A
is well
conditioned. Otherwise closeness will vary. See class docstring for details.
Examples:
# Make an operator acting like batch matrix A. Assume A.shape = [..., M, N]
operator = LinearOperator(...)
operator.shape = [..., M, N]
# Solve one linear system for every member of the batch.
RHS = ... # shape [..., M]
X = operator.solvevec(RHS)
# X is the solution to the linear system
# sum_j A[..., :, j] X[..., j] = RHS[..., :]
operator.matvec(X)
==> RHS
Args | |
---|---|
rhs
|
Tensor with same dtype as this operator.
rhs is treated like a [batch] vector meaning for every set of leading
dimensions, the last dimension defines a vector. See class docstring
for definition of compatibility regarding batch dimensions.
|
adjoint
|
Python bool . If True , solve the system involving the adjoint
of this LinearOperator : A^H X = rhs .
|
name
|
A name scope to use for ops added by this method. |
Returns | |
---|---|
Tensor with shape [...,N] and same dtype as rhs .
|
Raises | |
---|---|
NotImplementedError
|
If self.is_non_singular or is_square is False.
|
tensor_rank_tensor
tensor_rank_tensor(
name='tensor_rank_tensor'
)
Rank (in the sense of tensors) of matrix corresponding to this operator.
If this operator acts like the batch matrix A
with
A.shape = [B1,...,Bb, M, N]
, then this returns b + 2
.
Args | |
---|---|
name
|
A name for this Op .
|
Returns | |
---|---|
int32 Tensor , determined at runtime.
|
to_dense
to_dense(
name='to_dense'
)
Return a dense (batch) matrix representing this operator.
trace
trace(
name='trace'
)
Trace of the linear operator, equal to sum of self.diag_part()
.
If the operator is square, this is also the sum of the eigenvalues.
Args | |
---|---|
name
|
A name for this Op .
|
Returns | |
---|---|
Shape [B1,...,Bb] Tensor of same dtype as self .
|
with_name_scope
@classmethod
with_name_scope( method )
Decorator to automatically enter the module name scope.
class MyModule(tf.Module):
@tf.Module.with_name_scope
def __call__(self, x):
if not hasattr(self, 'w'):
self.w = tf.Variable(tf.random.normal([x.shape[1], 3]))
return tf.matmul(x, self.w)
Using the above module would produce tf.Variable
s and tf.Tensor
s whose
names included the module name:
mod = MyModule()
mod(tf.ones([1, 2]))
<tf.Tensor: shape=(1, 3), dtype=float32, numpy=..., dtype=float32)>
mod.w
<tf.Variable 'my_module/Variable:0' shape=(2, 3) dtype=float32,
numpy=..., dtype=float32)>
Args | |
---|---|
method
|
The method to wrap. |
Returns | |
---|---|
The original method wrapped such that it enters the module's name scope. |
__getitem__
__getitem__(
slices
)
__matmul__
__matmul__(
other
)