(1) A new benchmark, MV-RGBT, is collected to make it representative of multi-modal warranting scenarios, filling the gap between the data in current benchmarks and imaging conditions which motivate RGBT tracking.

(2) A new problem, `when to fuse', is posed to develop reliable fusion strategies for RGBT trackers, as in MMW scenarios multi-modal information fusion may be counterproductive. To facilitate its discussion, a new solution, MoETrack, with multiple tracking experts is proposed. It performs state-of-the-art on several benchmarks, including MV-RGBT, LasHeR, and VTUAV-ST.

(3) A new compositional perspective for method evaluation is provided by categorising MV-RGBT into two subsets, MV-RGBT-RGB and MV-RGBT-TIR, promoting a novel in-depth analysis and offering insightful recommendations for future developments in RGBT tracking.

🫵Find our survey work at repo

Data examples from MV-RGBT

Using a single modality in two typical MMW scenarios

• Multi-modal vs. single-modal
• RGB vs. TIR

• Data is availble at here with code TZYD
• Toolkit is available at here with code TZYD

⭐ ATTENTION：When testing, please follow 'test_Mydataset.py' to load the category information of MV-RGBT.

⭐ More detailed introduction of the proposed method, MoETrack, is available here