Face Photo-Sketch Synthesis Via Domain-Invariant Feature Embedding

Abstract

Face photo-sketch synthesis involves transforming photos into sketches and vice versa. A well-transformed image should preserve its original identity characteristics and naturalness. However, identity preservation remains a challenge because of the large discrepancy between the photo and sketch domains. To this end, we propose a novel face photo-sketch synthesis framework that uses domain-invariant feature embedding (DIFE). The DIFE framework generates images assuming the domain-invariant feature of an image pair for the same person to be the identity information. A joint feature embedding module considers latent features from two different domains as input and transfers them into the domain-invariant latent space. Subsequently, a semantic-aware decoder completes the desired image guided by multiscale facial parsing masks. Experimental results demonstrate that the DIFE method outperforms state-of-the-art approaches visually and perceptually.