Ashish Patel 🇮🇳’s Post

𝗗𝗮𝘆-𝟰𝟴𝟳 𝗖𝗼𝗺𝗽𝘂𝘁𝗲𝗿 𝗩𝗶𝘀𝗶𝗼𝗻 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 Barbershop: GAN-based Image Compositing using Segmentation Masks by KAUST (King Abdullah University of Science and Technology) Follow me for a similar post: Ashish Patel. ------------------------------------------------------------------- 𝗜𝗻𝘁𝗲𝗿𝗲𝘀𝘁𝗶𝗻𝗴 𝗙𝗮𝗰𝘁𝘀 : 🔸 This paper is published in arxiv 2021. 🔸 Hairstyle transfer is accomplished by transferring appearance (fine style attributes) and structure (coarse style attributes) from reference images into a composite image.  🔸 In each inset, the appearance, structure, and target masks for a hairstyle image are shown on the left, with the hair shape in magenta.  🔸 Inset(a) is a reference image used for the face and background, and (e) is a reconstruction using our novel 𝐹𝑆 latent space.  🔸 In (b) a reference image is used to transfer hair structure, but the hair’s appearance is from the original face, and (c) transfers both appearance and structure from a hair reference, in (d) and (f) both structure and appearance attributes are transferred, (g) and (h) use a hair shape that is different from any of the reference images. ------------------------------------------------------------------- 𝗜𝗠𝗣𝗢𝗥𝗧𝗔𝗡𝗖𝗘 👉 Seamlessly blending features from multiple images is extremely challenging because of complex relationships in lighting, geometry, and partial occlusion, which cause coupling between different parts of the image.  👉 Even though recent work on GANs enables the synthesis of realistic hair or faces, it remains difficult to combine them into a single, coherent, and plausible image rather than a disjointed set of image patches.  👉 We present a novel solution to image blending, particularly for the problem of hairstyle transfer, based on GAN-inversion.  👉 We propose a novel latent space for image blending that is better at preserving detail and encoding spatial information and propose a new GAN-embedding algorithm that can modify images to conform to a common segmentation mask slightly.  👉 Our novel representation enables the transfer of the visual properties from multiple reference images, including specific details such as moles and wrinkles, and because we do image blending in a latent space, we are able to synthesize coherent images.  👉 Our approach avoids blending artefacts present in other approaches and finds a globally consistent image.  👉 Our results demonstrate a significant improvement over the current state of the art in a user study, with users preferring our blending solution over 95 per cent of the time. #computervision #artificialintelligence #deeplearning #machinelearning #semanticsegmentation #technology