tfm.vision.layers.ROISampler

View source on GitHub

Samples ROIs and assigns targets to the sampled ROIs.

tfm.vision.layers.ROISampler(
    mix_gt_boxes: bool = True,
    num_sampled_rois: int = 512,
    foreground_fraction: float = 0.25,
    foreground_iou_threshold: float = 0.5,
    background_iou_high_threshold: float = 0.5,
    background_iou_low_threshold: float = 0,
    skip_subsampling: bool = False,
    **kwargs
)

Args
mix_gt_boxes	A bool of whether to mix the groundtruth boxes with proposed ROIs.
num_sampled_rois	An int of the number of sampled ROIs per image.
foreground_fraction	A float in [0, 1], what percentage of proposed ROIs should be sampled from the foreground boxes.
foreground_iou_threshold	A float that represents the IoU threshold for a box to be considered as positive (if >= foreground_iou_threshold).
background_iou_high_threshold	A float that represents the IoU threshold for a box to be considered as negative (if overlap in [background_iou_low_threshold, background_iou_high_threshold]).
background_iou_low_threshold	A float that represents the IoU threshold for a box to be considered as negative (if overlap in [background_iou_low_threshold, background_iou_high_threshold])
skip_subsampling	a bool that determines if we want to skip the sampling procedure than balances the fg/bg classes. Used for upper frcnn layers in cascade RCNN.
**kwargs	Additional keyword arguments passed to Layer.

Methods

call

View source

call(
    boxes: tf.Tensor, gt_boxes: tf.Tensor, gt_classes: tf.Tensor
)

Assigns the proposals with groundtruth classes and performs subsmpling.

Given proposed_boxes, gt_boxes, and gt_classes, the function uses the following algorithm to generate the final num_samples_per_image RoIs.

1. Calculates the IoU between each proposal box and each gt_boxes.
2. Assigns each proposed box with a groundtruth class and box by choosing the largest IoU overlap.
3. Samples num_samples_per_image boxes from all proposed boxes, and returns box_targets, class_targets, and RoIs.

Args
boxes	A tf.Tensor of shape of [batch_size, N, 4]. N is the number of proposals before groundtruth assignment. The last dimension is the box coordinates w.r.t. the scaled images in [ymin, xmin, ymax, xmax] format.
gt_boxes	A tf.Tensor of shape of [batch_size, MAX_NUM_INSTANCES, 4]. The coordinates of gt_boxes are in the pixel coordinates of the scaled image. This tensor might have padding of values -1 indicating the invalid box coordinates.
gt_classes	A tf.Tensor with a shape of [batch_size, MAX_NUM_INSTANCES]. This tensor might have paddings with values of -1 indicating the invalid classes.

Returns
sampled_rois	A tf.Tensor of shape of [batch_size, K, 4], representing the coordinates of the sampled RoIs, where K is the number of the sampled RoIs, i.e. K = num_samples_per_image.
sampled_gt_boxes	A tf.Tensor of shape of [batch_size, K, 4], storing the box coordinates of the matched groundtruth boxes of the samples RoIs.
sampled_gt_classes	A tf.Tensor of shape of [batch_size, K], storing the classes of the matched groundtruth boxes of the sampled RoIs.
sampled_gt_indices	A tf.Tensor of shape of [batch_size, K], storing the indices of the sampled groudntruth boxes in the original gt_boxes tensor, i.e., gt_boxes[sampled_gt_indices[:, i]] = sampled_gt_boxes[:, i].