Misaligned Image Integration With Local Linear Model

ABSTRACT:

We present a new image integration technique for a flash and long-exposure image pair to capture a dark scene without incurring blurring or noisy artifacts. Most existing methods require well-aligned images for the integration, which is often a burdensome restriction in practical use. We address this issue by locally transferring the colors of the flash images using a small fraction of the corresponding pixels in the long exposure images. We formulate the image integration as a convex optimization problem with the local linear model. The proposed method makes it possible to integrate the color of the long exposure image with the detail of the flash image without causing any harmful effects to its contrast, where we do not need perfect alignment between the images by virtue of our new integration principle. We show that our method successfully outperforms the state of the art in the image integration and reference-based color transfer for challenging misaligned data sets.

PROJECT OUTPUT VIDEO:

EXISTING SYSTEM:

• Many literatures discuss significant improvement in image quality by using an additional image such as a flash image. In existing systems, they combine the features of the images to integrate the colorfulness of a no-flash image with the vivid contrast of a flash image.

• Some of these methods have sufficient deblurring or denoising capabilities especially under dim lighting conditions, but they require perfectly aligned images. This is a severe restriction in practical use, since a camera needs to be fixed on a tripod, and a scene must be stationary.

• In existing systems, they propose color grading methods for a misaligned image pair. Their methods work well even for scenes with non-rigid motion, but they sometimes fail when the images have large lighting or color differences.

DISADVANTAGES OF EXISTING SYSTEM:

• Image restoration problems, such as denoising and deblurring, have been extensively studied in the past. Despite these successes, restoring a sharp image from a very noisy or blurred image remains a challenging problem.

• It is difficult to completely avoid blur and ringing artifacts. Furthermore, the quality of the estimation may be further degraded when the kernel is time-varying, and thus, these approaches can barely handle any blurring due to partial object motion.

• A significant drawback with this is that all of these methods require a perfectly aligned image pair, and even a little misalignment yields serious degradation in the result.

PROPOSED SYSTEM:

• We propose a new image integration approach for a misaligned image pair. In our framework, two kinds of images are used to restore an image with vivid colors and a high contrast under dim lighting conditions.

• One is a long-exposure image taken at a slow shutter speed and low ISO sensitivity. The image is blurry due to the low lighting illumination. The other image is a flash image, which is taken with a flash light and a faster shutter speed.

• The flash image has a sharper contrast, but contains unnatural color caused by the artificial light. The long-exposure image is taken at a slower shutter speed. In our framework, the image is blurry but has more natural color tone than the flash image.

• Our purpose is to reconstruct a high quality image from the aforementioned image set Flash and Long Exposure Image by combining their features. We start with the pixel matching to find correspondences between the two images. Our method uses only a sparse set of matched pixel pairs. Then we transform the color in the whole of the flash image based on the sparse set. This process ensures robustness to local illumination change, and it works well even if only a fraction of reliable matched pixel pairs is found.

ADVANTAGES OF PROPOSED SYSTEM:

• Our approach does not require a perfect alignment between the images by virtue of our new integration principle and can yield an image with natural colors and a sharp contrast even when the illumination in the images largely differ.

• Its performance is comparable to or even better than those existing methods.

• Our image integration yields images with vivid color from the high ISO image and the sharp detail of the flash image with little noise. Note that the conventional methods for flash/no-flash integration yield serious degradation in resultant images, as they handle only aligned images.

MODULES:

• Correspondence Search

• Pixel Selection

• Image Integration

MODULE DESCRIPTIONS:

Correspondence Search:

• In this stage, first input images such as flash and long exposure images are acquired from gallery.

• Given a flash/long-exposure image pair, we first search for the correspondences between the two images. There are many correspondence algorithms such as the patch-based and feature-based methods. Since in our case the twoimages contain different illuminations, the feature-based methods often perform better than the patch-based ones, so we adopt the SIFT flow to robustly detect an initial set of correspondences between the images.

• The SIFT flow is the dense correspondence algorithm, which is generally more reliable than a pixel-based matching such as the optical flow.We express the alignment from the long-exposure image to the flash image as Q^T:= TP(Q)

where TP(·) is the SIFT flow operator. The long-exposure image Q is transformed to by the SIFT flow so that Q^T is sufficiently close to Flash Image.

• Although the SIFT flow algorithm robustly finds the correspondences and fulfill a reliable image alignment, some misaligned pixels still remain. When the displacement between the two images is large, the resultant image easily suffers from the mismatches. Therefore, we select some reliable pixel pairs from the aligned guide image.

Pixel Selection:

• This is second stage of our method, in this pixel selection is performed.

• We select only a fraction of the matched pixels from the aligned image pair, which improves the robustness to local illumination change.

Image Integration:

• This is our final stage of our proposed system.

• Finally we integrate the two images to obtain a high quality image. The image integration problem is based on the local linear model (LLM). The optimization procedure integrates the color information of the long-exposure image and the details of the flash image while maintaining its local features.

SYSTEM REQUIREMENTS:

HARDWARE REQUIREMENTS:

• System            : Pentium Dual Core.

• Hard Disk       : 120 GB.

• Monitor          : 15’’ LED

• Input Devices : Keyboard, Mouse

• Ram                : 1GB.

SOFTWARE REQUIREMENTS:

• Operating system : Windows 7.

• Coding Language : MATLAB

• Tool                       : MATLAB R2013A

REFERENCE:

Tatsuya Baba, Student Member, IEEE, Ryo Matsuoka, Student Member, IEEE, Keiichiro Shirai, Member, IEEE, and Masahiro Okuda, Senior Member, IEEE, “Misaligned Image Integration With Local Linear Model”, IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 25, NO. 5, MAY 2016.