Warning: This project is deprecated. TensorFlow Addons has stopped development, The project will only be providing minimal maintenance releases until May 2024. See the full announcement here or on github.


Image warping using correspondences between sparse control points.

Apply a non-linear warp to the image, where the warp is specified by the source and destination locations of a (potentially small) number of control points. First, we use a polyharmonic spline (tfa.image.interpolate_spline) to interpolate the displacements between the corresponding control points to a dense flow field. Then, we warp the image using this dense flow field (tfa.image.dense_image_warp).

Let t index our control points. For regularization_weight = 0, we have: warped_image[b, dest_control_point_locations[b, t, 0], dest_control_point_locations[b, t, 1], :] = image[b, source_control_point_locations[b, t, 0], source_control_point_locations[b, t, 1], :].

For regularization_weight > 0, this condition is met approximately, since regularized interpolation trades off smoothness of the interpolant vs. reconstruction of the interpolant at the control points. See tfa.image.interpolate_spline for further documentation of the interpolation_order and regularization_weight arguments.

image Either a 2-D float Tensor of shape [height, width], a 3-D Tensor of shape [height, width, channels], or a 4-D Tensor of shape [batch_size, height, width, channels]. batch_size is assumed as one when image is a 2-D or 3-D Tensor.
source_control_point_locations [batch_size, num_control_points, 2] float Tensor.
dest_control_point_locations [batch_size, num_control_points, 2] float Tensor.
interpolation_order polynomial order used by the spline interpolation
regularization_weight weight on smoothness regularizer in interpolation
num_boundary_points How many zero-flow boundary points to include at each image edge. Usage:

  • num_boundary_points=0: don't add zero-flow points
  • num_boundary_points=1: 4 corners of the image
  • num_boundary_points=2: 4 corners and one in the middle of each edge (8 points total)
  • num_boundary_points=n: 4 corners and n-1 along each edge
name A name for the operation (optional).

Note that image and offsets can be of type tf.half, tf.float32, or tf.float64, and do not necessarily have to be the same type.

warped_image a float Tensor with the same shape and dtype as image.
flow_field [batch_size, height, width, 2] float Tensor containing the dense flow field produced by the interpolation.