Skip to content

Feature request: Support for aug_test. #138

@ypwhs

Description

@ypwhs

Difference of mmdetection and mmdeploy at aut_test

Now mmdeploy is to directly take the first image prediction, which is different from the behavior of mmdetection: https://github.com/open-mmlab/mmdeploy/blob/v0.2.0/mmdeploy/codebase/mmdet/deploy/object_detection_model.py#L191

input_img = img[0].contiguous()
outputs = self.forward_test(input_img, img_metas, *args, **kwargs)

mmdetction will combine the results of multiple scale results and do nms: https://github.com/open-mmlab/mmdetection/blob/v2.20.0/mmdet/models/roi_heads/test_mixins.py#L138-L176

def aug_test_bboxes(self, feats, img_metas, proposal_list, rcnn_test_cfg):
    """Test det bboxes with test time augmentation."""
    aug_bboxes = []
    aug_scores = []
    for x, img_meta in zip(feats, img_metas):
        # only one image in the batch
        img_shape = img_meta[0]['img_shape']
        scale_factor = img_meta[0]['scale_factor']
        flip = img_meta[0]['flip']
        flip_direction = img_meta[0]['flip_direction']
        # TODO more flexible
        proposals = bbox_mapping(proposal_list[0][:, :4], img_shape,
                                 scale_factor, flip, flip_direction)
        rois = bbox2roi([proposals])
        bbox_results = self._bbox_forward(x, rois)
        bboxes, scores = self.bbox_head.get_bboxes(
            rois,
            bbox_results['cls_score'],
            bbox_results['bbox_pred'],
            img_shape,
            scale_factor,
            rescale=False,
            cfg=None)
        aug_bboxes.append(bboxes)
        aug_scores.append(scores)
    # after merging, bboxes will be rescaled to the original image size
    merged_bboxes, merged_scores = merge_aug_bboxes(
        aug_bboxes, aug_scores, img_metas, rcnn_test_cfg)
    if merged_bboxes.shape[0] == 0:
        # There is no proposal in the single image
        det_bboxes = merged_bboxes.new_zeros(0, 5)
        det_labels = merged_bboxes.new_zeros((0, ), dtype=torch.long)
    else:
        det_bboxes, det_labels = multiclass_nms(merged_bboxes,
                                                merged_scores,
                                                rcnn_test_cfg.score_thr,
                                                rcnn_test_cfg.nms,
                                                rcnn_test_cfg.max_per_img)
    return det_bboxes, det_labels

Feature request

Using multi-scale prediction can improve the accuracy for oversized and undersized objects. I want mmdeploy to implement the same aug_test as mmdetection.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions