Ashish Patel 🇮🇳’s Post

𝗗𝗮𝘆-𝟮𝟰𝟭 𝗖𝗼𝗺𝗽𝘂𝘁𝗲𝗿 𝗩𝗶𝘀𝗶𝗼𝗻 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 𝗔𝗶𝗿𝗯𝗲𝗿𝘁: 𝗜𝗻-𝗱𝗼𝗺𝗮𝗶𝗻 𝗣𝗿𝗲𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝗳𝗼𝗿 𝗩𝗶𝘀𝗶𝗼𝗻-𝗮𝗻𝗱-𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗡𝗮𝘃𝗶𝗴𝗮𝘁𝗶𝗼𝗻 𝗯𝘆 𝗜𝗜𝗜𝗧 𝗛𝘆𝗱𝗲𝗿𝗮𝗯𝗮𝗱, 𝗜𝗻𝗱𝗶𝗮 Follow me for a similar post: 🇮🇳 Ashish Patel Interesting Facts : 🔸 This paper is published ICCV2021. ------------------------------------------------------------------- 𝗔𝗺𝗮𝘇𝗶𝗻𝗴 𝗥𝗲𝘀𝗲𝗮𝗿𝗰𝗵 : https://lnkd.in/eKx-EMPa Dataset: https://lnkd.in/eQiQ6sRc ------------------------------------------------------------------- 𝗜𝗠𝗣𝗢𝗥𝗧𝗔𝗡𝗖𝗘 🔸 Vision-and-language navigation (VLN) aims to enable embodied agents to navigate in realistic environments using natural language instructions. Given the scarcity of domain-specific training data and the high diversity of image and language inputs, the generalization of VLN agents to unseen environments remains challenging.  🔸 Recent methods explore pretraining to improve generalization, however, the use of generic image-caption datasets or existing small-scale VLN environments is suboptimal and results in limited improvements.  🔸 In this work, we introduce BnB, a large-scale and diverse in-domain VLN dataset. We first collect image-caption (IC) pairs from hundreds of thousands of listings from online rental marketplaces. Using IC pairs we next propose automatic strategies to generate millions of VLN path-instruction (PI) pairs.  🔸 We further propose a shuffling loss that improves the learning of temporal order inside PI pairs. We use BnB pretrain our Airbert model that can be adapted to discriminative and generative settings and show that it outperforms state of the art for Room-to-Room (R2R) navigation and Remote Referring Expression (REVERIE) benchmarks. Moreover, our in-domain pretraining significantly increases performance on a challenging few-shot VLN evaluation, where we train the model only on VLN instructions from a few houses. #computervision #artificialintelligence #machinelearning