Motivation

See the discussion in open-mmlab/mmdetection#7020. In MMDet, MMSeg, and other downstream repos, we wish to directly use backbones supported by TIMM in MMCls. Therefore, it is necessary to support this option for downstream tasks.
This PR will close #665.
This PR (especially test_timm_backbone_features_only) is based on open-mmlab/mmsegmentation#998.

Modification

• update TIMMBackbone to enable extracting feature pyramid (multi-scale feature maps) with features_only=True
• update and enhance docstring and log messages
• add unit tests for features_only=True
• fix a minor bug in test_timm_backbone

Use cases (Optional)

MMDetection

Here is an example config retinanet_timm_tv_resnet50_fpn_fp16_4x4_1x_coco.py.
The results at epoch 1 (bbox_mAP_copypaste: 0.162 0.285 0.164 0.084 0.201 0.201) are similar to those of retinanet_r50_fpn_fp16_1x_coco.py (bbox_mAP_copypaste: 0.164 0.284 0.168 0.082 0.189 0.203).

_base_ = [
    '../_base_/models/retinanet_r50_fpn.py',
    '../_base_/datasets/coco_detection.py',
    '../_base_/schedules/schedule_1x.py', '../_base_/default_runtime.py'
]

# import to trigger register_module in mmcls
custom_imports = dict(imports=['mmcls.models'], allow_failed_imports=False)
model = dict(
    backbone=dict(
        _delete_=True,
        type='mmcls.TIMMBackbone',
        model_name='tv_resnet50',  # ResNet-50 with torchvision weights
        features_only=True,
        pretrained=True,
        out_indices=(1, 2, 3, 4)))

optimizer = dict(type='SGD', lr=0.01, momentum=0.9, weight_decay=0.0001)

# disable NumClassCheckHook
custom_hooks = []

data = dict(samples_per_gpu=4)

fp16 = dict(loss_scale=dict(init_scale=512))

MMSegmentation

Here is an example config upernet_timm_resnet50d_512x512_20k_voc12aug.py.
mIoU 69.75 at 2000 iter. umm... too high? I'm not familiar with mmseg, and the config may be wrong.
In any case, training and evaluation work.

_base_ = [
    '../_base_/models/upernet_r50.py',
    '../_base_/datasets/pascal_voc12_aug.py', '../_base_/default_runtime.py',
    '../_base_/schedules/schedule_20k.py'
]

# import to trigger register_module in mmcls
custom_imports = dict(imports=['mmcls.models'], allow_failed_imports=False)
model = dict(
    pretrained=None,
    backbone=dict(
        _delete_=True,
        type='mmcls.TIMMBackbone',
        model_name='resnet50d',  # instead of ResNet-50-C
        features_only=True,
        pretrained=True,
        out_indices=(1, 2, 3, 4),
        norm_layer='SyncBN'),
    decode_head=dict(num_classes=21),
    auxiliary_head=dict(num_classes=21))

Checklist

Before PR:

• Pre-commit or other linting tools are used to fix the potential lint issues.
• Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
• The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
• The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

• If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects, like MMDet or MMSeg.
• CLA has been signed and all committers have signed the CLA in this PR.