Nighttime Vehicle Detection Based on Bio-Inspired Image Enhancement and Weighted Score-Level Feature Fusion

ABSTRACT:

This paper presents an effective nighttime vehicle detection system that combines a novel bio-inspired image enhancement approach with a weighted feature fusion technique. Inspired by the retinal mechanism in natural visual processing, we develop a nighttime image enhancement method by modeling the adaptive feedback from horizontal cells and the center-surround antagonistic receptive fields of bipolar cells. Furthermore, we extract features based on the convolutional neural network, histogram of oriented gradient, and local binary pattern to train the classifiers with support vector machine. These features are fused by combining the score vectors of each feature with the learnt weights. During detection, we generate accurate regions of interest by combining vehicle taillight detection with object proposals. Experimental results demonstrate that the proposed bio-inspired image enhancement method contributes well to vehicle detection. Our vehicle detection method demonstrates a 95.95% detection rate at 0.0575 false positives per image and outperforms some state-of-the-art techniques. Our proposed method can deal with various scenes including vehicles of different types and sizes and those with occlusions and in blurred zones. It can also detect vehicles at various locations and multiple vehicles.

PROJECT OUTPUT VIDEO:

EXISTING SYSTEM:

• They studied the question of feature sets for robust visual object recognition; adopting linear SVM based human detection as a test case.

• After reviewing existing edge and gradient based descriptors, they showed experimentally that grids of histograms of oriented gradient (HOG) descriptors significantly outperform previous feature sets for human detection.

• They studied the influence of each stage of the computation on performance, concluding that fine-scale gradients, fine orientation binning, relatively coarse spatial binning, and high-quality local contrast normalization in overlapping descriptor blocks are all important for good results.

DISADVANTAGES OF EXISTING SYSTEM:

• Detection methods based on single features have been proved to be effective. However, when dealing with more complex scenes, these types of detection methods might lead to misclassifications.

PROPOSED SYSTEM:

• Our proposed method has two stages such as training stage and detection stage. During the training stage, the original training samples are enhanced by the proposed image enhancement approach and then three complementary features: CNN features, HOG and LBP are extracted from the enhanced images. Three SVM classifiers are trained with LibSVM and five-fold cross-validation is carried out using each individual feature respectively.

• The score vectors of each classifier are also computed. The scores are used to learn weights of each feature for each individual class using a linear SVM. During the detection stage, accurate ROIs are extracted from the input images. Subsequently, the three features are extracted from each enhanced ROI.

• Next, the trained classifiers are used to classify the corresponding features and compute scores which are summed with learnt weights to obtain final score vectors for prediction. Finally, post-processing is performed to ensure that each vehicle is surrounded by a single window.

• This paper makes the following contributions:

• A novel and effective bio-inspired nighttime image enhancement method is proposed to enhance the contrast, the brightness and details of nighttime images. Our approach is more effective than state-of-the-art methods.

• To complement CNN features, we extract HOG and LBP, and then utilize an effective weighted score-level feature fusion based on learnt weights using a linear SVM to combine the three features. The way to learn weights and bias terms is different from state-of-the-art classifier ensemble methods.

ADVANTAGES OF PROPOSED SYSTEM:

• The benefits of our score-level feature fusion based on learnt weights are as follows:

• The strength of complementarity of each classifier (i.e., feature) for the individual class is fully utilized. For the same classifier, the weights of each class are different and learnt separately. Bias terms are learnt to calibrate the original scores. These aspects are neglected in some score-level feature fusion approaches.

• Our proposed feature fusion method can be easily used in multi-class detection tasks. Because the weight and bias term of each classifier for each class are learnt separately, when dealing with more features and more classes we only need to learn new weights and bias terms for new additional classes and features, and sum the new calibrated scores with the existing scores.

• Because each feature is used to train a classifier and the dimensionality of each feature is unchanged, our feature fusion approach can avoid the curse of dimensionality as well.

• We find that our method successfully detects the vehicles in various numbers, types, locations and sizes from various scenes.

MODULES:

1. Image Acquisition
2. Preprocessing
3. Enhancement
4. Detection
5. Validation of Feature Complementarity

MODULES DESCRIPTION:

1. Image Acquisition:

• Image acquisition in image processing can be broadly defined as the action of retrieving an image from some source, usually a hardware-based source, so it can be passed through whatever processes need to occur afterward.

• Performing image acquisition in image processing is always the first step in the workflow sequence because, without an image, no processing is possible. The image that is acquired is completely unprocessed and is the result of whatever hardware was used to generate it, which can be very important in some fields to have a consistent baseline from which to work.

• Reference image and test images are acquired from public repository.

2. Preprocessing:

In preprocessing stage, frames extraction and gray image generation is proposed. Retinal information processing mechanism begins with the sampling by rod and cone photoreceptors. The red (R), green (G), and blue (B) components of the input image are processed respectively by long-, medium-, and short- wavelength cone photoreceptors of the retina. Given an input image I(x, y), it is first sampled by three types of cone photoreceptors, that is, converted to R, G, and B channel images (i.e., Ic(x, y), c ∈ (R,G, B)). Then the horizontal cells (HCs) collect three cone signals from photoreceptors

3. Enhancement:

The third step is image enhancement. For nighttime vehicle detection, nighttime image enhancement is a necessary pre-processing step. We follow the ROI extraction framework but change the image enhancement to our proposed bio-inspired nighttime image enhancement method. First, the possible vehicle taillight regions are detected using the following steps: (1) converting the RGB images to intensity images and reducing noise using an empirical threshold (2) estimating the images

4. Detection

Our final stage is night time vehicle detection. Our proposed feature fusion method can be easily used in multi-class detection tasks. Because the weight and bias term of each classifier for each class are learnt separately, when dealing with more features and more classes we only need to learn new weights and bias terms for new additional classes and features, and sum the new calibrated scores with the existing scores.

5. Validation of Feature Complementarity

In this paper, to complement CNN features we extract HOG and LBP. We compared miss rate vs. FPPI curves of the proposed fusion and using each single feature. We validated the complementarity of the three features and the effectiveness of our weighted feature fusion in Fig. 6. In these comparisons, the differences are the features used, the other parts, i.e., image enhancement, ROI extraction and detection method are the same. From Fig. 6, CNN is better than HOG and LBP and the proposed fusion of CNN, HOG and LBP is better than using three single features. These results demonstrate that HOG and LBP are complements of CNN and our proposed fusion make full use of the complementarity of multiple features.

SYSTEM REQUIREMENTS:

HARDWARE REQUIREMENTS:

• System           : Pentium Dual Core.

• Hard Disk      : 120 GB.

• Monitor          : 15’’ LED

• Input Devices : Keyboard, Mouse

• Ram                : 1GB.

SOFTWARE REQUIREMENTS:

• Operating system : Windows 7.

• Coding Language : MATLAB

• Tool                       : MATLAB R2013A

REFERENCE:

Hulin Kuang, Xianshi Zhang, Yong-Jie Li, Member, IEEE, Leanne Lai Hang Chan, Member, IEEE, and Hong Yan, Fellow, IEEE, “Nighttime Vehicle Detection Based on Bio-Inspired Image Enhancement and Weighted Score-Level Feature Fusion”, IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, 2017.