DeepFaceDrawing : Deep Learning to Create Realistic Face Images from Sketches

To combat overfitting and the requirement for expertly made sketches, the authors of DeepFaceDrawing have presented a revolutionary picture synthesis sketch. The key concept is to implicitly learn a space of plausible face sketches from real face sketch images and find the closest point in this space (using manifold projection) to approximate an input sketch. This is different from the traditional idea of using hard constraints to guide image synthesis. This makes it possible for the proposed work to create high-quality face photographs even from incomplete or imperfect sketches.

Local to global approach : Due to a lack of training data, it is not possible to learn a space of realistic face sketches globally. Thus, the authors suggested that important face components feature embeddings be learned. The mouth, nose, and eyes are some of these essential parts. pushing the matching components in the input sketch in the direction of the discovered underlying component manifolds is the goal here.

Using a novel deep neural network, realistic images can be generated from embedded component features, with multi-channel feature maps serving as intermediate outputs to enhance information flow.

Overview of the Method Proposed

The model framework, which enables high-quality sketch-to-image translation, will be brief covered in this session. The three primary components of the framework are component embedding, feature mapping, and image synthesis. This framework receives two stages of training. Let's go right into this framework's details.

What Is Image-to-Image Translation?

A deep Learning job in computer vision called "image-to-image translation" aims to learn the relationship between an input image and an output image. The process of translating one potential representation of a scene into another representation is known as image-to-image translation.

Fast production of face pictures from freehand sketches is now possible because to recent advancements in image-to-image translation. All of these methods, nevertheless, quickly outperform input sketches. As a result, it necessitates professionally created sketches, which restricts the population of users of applications based on these methods.

These deep learning-based algorithms attempt to infer missing texture and shading information using an input sketch as a hard constraints. Thus, the issue is defined as a reconstruction problem. Additionally, these models are trained using edge maps and pairs of actual photos. For this reason, test images must be of an edge map-like quality.

Conclusions

A unique deep learning architecture for creating realistic face images from rough or incomplete freehand sketches. Using a local-to-global approach, we first break down a sketched face into its component parts, then refine each part by projecting it to a component manifold defined by the component samples already present in the feature spaces, maps the refined feature vector to the feature maps for spatial combination, and finally translate the combined feature maps to realistic images.