Ashish Patel 🇮🇳’s Post

𝗗𝗮𝘆-𝟰𝟲𝟵 𝗖𝗼𝗺𝗽𝘂𝘁𝗲𝗿 𝗩𝗶𝘀𝗶𝗼𝗻 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 Masked Siamese Networks for Label-Efficient Learning by Facebook Follow me for a similar post: Ashish Patel ------------------------------------------------------------------- 𝗜𝗻𝘁𝗲𝗿𝗲𝘀𝘁𝗶𝗻𝗴 𝗙𝗮𝗰𝘁𝘀 : 🔸 This paper is published Arxiv2022. 🔸 Official: https://lnkd.in/gKZY9XJr ------------------------------------------------------------------- 𝗜𝗠𝗣𝗢𝗥𝗧𝗔𝗡𝗖𝗘 🌻 We propose Masked Siamese Networks (MSN), a self-supervised learning framework for learning image representations. 🌷 Our approach matches the representation of an image view containing randomly masked patches to the representation of the original unmasked image. 🌹 This self-supervised pre-training strategy is particularly scalable when applied to Vision Transformers since only the unmasked patches are processed by the network. 🌺 As a result, MSNs improve the scalability of joint-embedding architectures, while producing representations of a high semantic level that perform competitively on low-shot image classification. ☘️ For instance, on ImageNet-1K, with only 5,000 annotated images, our base MSN model achieves 72.4% top-1 accuracy, and with 1% of ImageNet-1K labels, we achieve 75.7% top-1 accuracy, setting a new state-of-the-art for self-supervised learning on this benchmark #computervision #artificialintelligence #technology