Facial Sketch Synthesis Using Two-dimensional Direct Combined Model-based Face-Specific Markov Network

ABSTRACT:

A facial sketch synthesis system is proposed featuring a two-dimensional direct combined model (2DDCM)-based face specific Markov network. In contrast to existing facial sketch synthesis systems, the proposed scheme aims to synthesize sketches which reproduce the unique drawing style of a particular artist, where this drawing style is learned from a dataset consisting of a large number of image/sketch pairwise training samples. The synthesis system comprises three modules, namely a global module, a local module, and an enhancement module. The global module applies a 2DDCM approach to synthesize the global facial geometry and texture of the input image. The detailed texture is then added to the synthesized sketch in a local patch-based manner using a parametric 2DDCM model and a non-parametric Markov random field (MRF) network. Notably, the MRF approach gives the synthesized results an appearance more consistent with the drawing style of the training samples, while the 2DDCM approach enables the synthesis of outcomes with a more derivative style. As a result, the similarity between the synthesized sketches and the input images is greatly improved. Finally, a post-processing operation is performed to enhance the shadowed regions of the synthesized image by adding strong lines or curves to emphasize the lighting conditions. The experimental results confirm that the synthesized facial images are in good qualitative and quantitative agreement with the input images as well as the ground-truth sketches provided by the same artist. The representing power of the proposed framework is demonstrated by synthesizing facial sketches from input images with a wide variety of facial poses, lighting conditions, and races even when such images are not included in the training dataset. Moreover, the practical applicability of the proposed framework is demonstrated by means of automatic facial recognition tests.

PROJECT OUTPUT VIDEO:

EXISTING SYSTEM:

• Existing sketch synthesis methods can be broadly classified as either image-based or exemplar-based. Methods of the former type generate sketch strokes directly based on the gray value or edge information contained in the input image.

• Generally speaking, the synthesis results produced by the parametric-based approaches described are derived directly from the training samples, e.g., the results are obtained by applying a linear combination process to the training samples.

• In summary, exemplar-based approaches, which model the correlation between the images and the sketches in a parametric-based way, tend to generate images with a blurred and more derivative-type appearance. By contrast, nonparametric-based methods, such as MRF networks, tend to produce clearer output images, but are sensitive to texture variations in the training dataset.

DISADVANTAGES OF EXISTING SYSTEM:

• They are unable to synthesize sketches in a specific drawing style (i.e., as can be achieved by a talented artist).

• When the unknown test image differs substantially from the training images (i.e., in terms of its appearance, pose, illumination, race, and so on), the synthesis quality tends to be degraded since suitable sketch patch candidates cannot be found among the training samples.

PROPOSED SYSTEM:

• The system comprises three modules, namely a global structure synthesis module, a local detailed texture synthesis module, and a synthesis result enhancement module.

• In the global module, a global 2DDCM model is used to construct the global facial geometry and texture of the input image. In the local module, a local 2DDCM model and MRF network are used to add local texture details to the global synthesized sketch.

• Notably, the local synthesis process is performed at the patch level, where each synthesized patch is selected either in a non-parametric fashion, i.e., the patch is selected from the training samples directly, or in a parametric manner, i.e., the patch is selected from among the 2DDCM synthesized patches. Hence, the local synthesized results contain both specific strokes from the training samples and virtual textures which differ from those of the training samples.

• As a result, the proposed framework provides the ability not only to accurately reproduce the distinctive sketching features of the input image in the unique style of a particular artist, but also to synthesize facial sketches with poses, gaze directions, and expressions not featured within the original training images.

ADVANTAGES OF PROPOSED SYSTEM:

• The 2DDCM kernel method used in the proposed framework has three main advantages over existing kernel methods.

• First, in contrast to parametric-based facial synthesis methods such as those presented, the 2DDCM algorithm models the facial structure correlation between the pairwise image and sketch samples in a single combined subspace. Thus, the correlation between them is better preserved. Moreover, compared to facial synthesis methods in which the images and sketches are represented in the form of one-dimensional vectors, the 2DDCM approach combines each image and sketch in a single two-dimensional matrix in order to better describe the facial geometry.

• Second, compared to existing MRF synthesis frameworks, such as that presented, the 2DDCM approach provides a more correct facial geometry structure for constraining the MRF process.

• Furthermore, the 2DDCM approach increases the number of derivative sketch candidates, and therefore overcomes the traditional sensitivity of MRF approaches to texture variation in the training dataset.

MODULES:

• Input Image

• Gray Image

• Patch extraction

• 2D DCM/Bilateral filter

• MRF

• Fusion

• Nearest Neighbour

MODULES DESCSRIPTION:

Input Image:

• In the first module, we develop the Input Image functionality. To select the path and file name given format from test image or else any folder .this process concept we used (imread ) command used.

Gray Image:

• In photography and computing, a grayscale or greyscale digital image is an image in which the value of each pixel is a single sample, that is, it carries only intensity information. Images of this sort, also known as black-and-white, are composed exclusively of shades of gray, varying from black at the weakest intensity to white at the strongest.

• Grayscale images are distinct from one-bit bi-tonal black-and-white images, which in the context of computer imaging are images with only two colors, black and white (also called bilevel or binary images). Grayscale images have many shades of gray in between.

• Grayscale images are often the result of measuring the intensity of light at each pixel in a single band of the electromagnetic spectrum (e.g. infrared, visible light, ultraviolet, etc.), and in such cases they are monochromatic proper when only a given frequency is captured. But also they can be synthesized from a full color image; see the section about converting to grayscale.

Patch Extraction:

• In patch extraction to be spilt into original image to be row and column wise patches

• That will be depend upon a pixel to compress the method and analysis the patched to be dictionary format store the values .

• After patch will be separate to be block images for the given row and column element.

• There is no universal or exact definition of what constitutes a feature, and the exact definition often depends on the problem or the type of application.

• Given that, a feature is defined as an “interesting” part of an image, and features are used as a starting point for many computer vision algorithms.

• Since features are used as the starting point and main primitives for subsequent algorithms, the overall algorithm will often only be as good as its feature detector

Bilateral Filter:

• A bilateral filter is a non-linear, edge-preserving and noise-reducing smoothing filter for images. The intensity value at each pixel in an image is replaced by a weighted average of intensity values from nearby pixels. This weight can be based on a Gaussian distribution. Crucially, the weights depend not only on Euclidean distance of pixels, but also on the radiometric differences (e.g. range differences, such as color intensity, depth distance, etc.). This preserves sharp edges by systematically looping through each pixel and adjusting weights to the adjacent pixels accordingly.

Markov random field:

• In the domain of physics and probability, a Markov random field (often abbreviated as MRF), Markov network or undirected graphical model is a set of random variables having a Markov property described by an undirected graph. In other words, a random field is said to be Markov random field if it satisfies Markov properties. A Markov network or MRF is similar to a Bayesian network in its representation of dependencies; the differences being that Bayesian networks are directed and acyclic, whereas Markov networks are undirected and may be cyclic. Thus, a Markov network can represent certain dependencies that a Bayesian network cannot (such as cyclic dependencies); on the other hand, it can’t represent certain dependencies that a Bayesian network can (such as induced dependencies). The underlying graph of a Markov random field may be finite or infinite.

Fusion:

• In computer vision, Multi sensor Image fusion is the process of combining relevant information from two or more images into a single image The resulting image will be more informative than any of the input images In remote sensing applications, the increasing availability of space borne sensors gives a motivation for different image fusion algorithms. Several situations in image processing require high spatial and high spectral resolution in a single image. Most of the available equipment is not capable of providing such data convincingly. Image fusion techniques allow the integration of different information sources. The fused image can have complementary spatial and spectral resolution characteristics. However, the standard image fusion techniques can distort the spectral information of the multispectral data while merging.

Nearest Neighbour:

• Nearest neighbor search (NNS), also known as proximity search, is the optimization problem of finding the point in a given set that is closest (or most similar) to a given point. Closeness is typically expressed in terms of a dissimilarity function: the less similar the objects, the larger the function values. Formally, the nearest-neighbor (NN) search problem is defined as follows: given a set S of points in a space M and a query point q ∈ M, find the closest point in S to q. Various solutions to the NNS problem have been proposed. The quality and usefulness of the algorithms are determined by the time complexity of queries as well as the space complexity of any search data structures that must be maintained. The informal observation usually referred to as the curse of dimensionality states that there is no general-purpose exact solution for NNS in high-dimensional Euclidean space using polynomial preprocessing and poly logarithmic search time.

SYSTEM REQUIREMENTS:

HARDWARE REQUIREMENTS:

• System           : Pentium Dual Core.

• Hard Disk      : 120 GB.

• Monitor          : 15’’ LED

• Input Devices : Keyboard, Mouse

• Ram                : 1GB.

SOFTWARE REQUIREMENTS:

• Operating system : Windows 7.

• Coding Language : MATLAB

• Tool                      : MATLAB R2013A

REFERENCE:

Ching-Ting Tu*, Yu-Hsien Chan, and Yi-Chung Chen, “Facial Sketch Synthesis Using Two-dimensional Direct Combined Model-based Face-Specific Markov Network”, IEEE Transactions on Image Processing 2016.