Face Synthesis from Sketches via GAN Fusion of Attention and Parametric tanh

Authors: Sun, J., Yu, H. and Jianjun, Z.

Journal: IEEE Transactions on Image Processing

Publisher: IEEE

ISSN: 1057-7149

Abstract:

Despite the extensive progression in face photo-sketch synthesis, there are few methods focusing on generating a color face image from a sketch. The existing methods pay less attention to learning the illumination or highlight distribution on the face region. However, the illumination is the key factor that makes the generated color face image look vivid and realistic. Moreover, the existing methods tend to employ the complicated image preprocessing and facial region patching approaches to generate the high-quality face images, which results in difficulties in implementation. In this paper, we propose a novel end-to-end generative adversarial network based model, tGAN, which utilizes a special tanh function. We present a Parametric tanh function to efficiently learn illumination distribution over the face. The proposed tGAN model consists of two U-Net generators and a discriminator. In particular, one of the U-Net generator integrates an attention mechanism to maintain the identity consistency and the correctness of facial texture for the generated face images. Moreover, we propose the first evaluation scheme to qualitatively and quantitatively assess the identity consistency and quality of the synthesized face images. The experimental results demonstrate the proposed tGAN outperforms the state-of-the-art methods. Particularly, the tGAN has a good generalization ability. It can further be used in the benchmarking test of facial spoofing attacks

Source: Manual