CarsTracking and Counting at Night
Yuan Been Chen
Department of Electronic Engineering, Chienkuo Technology University, Changhua City, Taiwan
Keywords:Moving object counting, background subtraction, edge-following scheme,motion connection.
Abstract. This work proposes a robust scheme to automaticallytracking and countingcars in the traffic surveillance. In the proposed method, pixels at a specific position of successive image frames are first processed by the modified iterative threshold selection technique to establish the background model. Second, an original image is subtracted by this background to obtain a difference image that is performed with the differential image between an original image and its precedent neighboring imageto yield an image with initial contour points of moving objects. Third, the robust edge-following scheme manipulates thesecontour points to produce closed-form objects.Particularly, two headlights of a car are merged with their corresponding reflective lights on the ground to yield two light objects for a car extraction at night. As compared to the conventional methods, the proposed method is demonstrated to have the best accuracy of moving object extraction.Finally, object motion connection iseffectively employed to track object paths and compute the number of moving cars.The practical implementation reveals that the proposed method can precisely and reliably estimate a traffic amount.
Introduction
Object extraction and tracking arefundamental tasks within the field of video processing. Many object extraction methods have been proposed in video applications such as tracking,surveillance, recognition and indexing [1]-[7]. Particularly, background subtraction is a commonly-used approach to detect and extract moving objects in a video sequence. This technique has been extensively investigated and applied for years. Particularly, a mixture of Gaussians (MoG) was proposed for the background subtraction [8] and efficient updating manners were developed [9]. In [8], Stauffer et al. employed a mixture of Gaussians which utilized a mixture of normal distributions to model a multimodal background image sequence. In [9], Zivkovicet al. presented recursive equations that were used to constantly update the parameters of a Gaussian mixture model (GMM). In [10], Elgammal et al. proposed a nonparametric kernel density estimation (KDE) method to build statistical representations associated with the scene background and foreground regions. However, the probability density functions of the background and foreground are likely to vary from image to image and may not have a known parametric form.Optical flow is another type of moving object extraction technique. It defines the translation of each pixel in a region where brightness constancy of corresponding pixels in consecutive frames is assumed. Popular techniques associated with optical flow were proposed by Horn et al. [11] and Lucas et al. [12]. Optical flow indicates pixel motions between a pair of images. To find out correct pixel motions is difficult because of the aperture problem and insufficient sampling near occlusion boundaries [13].
The goal of object tracking is to generate the trajectory of an object by locating its position in each frame of a video sequence. Tracking moving targets is achieved by comparing the extracted features and measuring the minimal distance between two temporal images.Object tracking methods can be classified into five categories: model-based, appearance-based, contour- and mesh-based, feature-based, and hybrid methods [14]. The problem of tracking curves in dense visual clutter is challenging. Kalman filtering [15] is inadequate because it is based on Gaussian densities which, being unimodal, cannot represent simultaneous alternative hypotheses.The Condensation algorithm [16] uses “factored sampling”, previously applied to the interpretation of static images, in which the probability distribution of possible interpretations is represented by a randomly generated set. Condensation uses learned dynamical models, together with visual observations, to propagate the random set over time. A Spatial-color Mixture of Gaussians (SMOG) and a SMOG-based similarity measure are proposed by Wang et al. [17]. A complete SMOG tracking algorithm can successfully track objects in many difficult situations.
In our tracking approach, the objects are represented using the appearance model. The centroid coordinate and gray-level mean of each object are computed. The tasks of detecting an object and establishing correspondence between the object instances across frames can either be performed separately or jointly. Finally, the connected component, illuminance constancy and motion connection are effectively applied to track object paths and compute the number of moving objects. The practical implementation reveals that the proposed method can precisely and reliably estimate the traffic amount.
Proposed method of moving object extraction
This work develops a method of object extraction and counting. Figure 1 shows the flow chart of the proposed object extraction scheme. Pixels at all positions employ such an iterative technique to establish the background model, as shown in Fig.1(b). The original image shown in Fig. 1(a) is subtracted by the background in Fig. 1(b) to obtain an absolute difference image, as shown in Fig.1(c). The modified edge-following scheme is conducted at the Region Of Interest (ROI) in Fig. 1(d) to yield closed-form objects with consideration of background contours. The closed-form objects are then performed by a fill operator to produce complete objects, as shown in Fig. 1(e) [18].
Figure 1.Flow chart of the proposed object extraction method and the output components of each block.
Proposed method of moving object counting
Two headlights are grouped to be one car. The region of the reflected light R(i, t) is merged with the closest headlights by using the threshold. Fig. 2shows the 17th frame of “traffic” video sequence. Fig. 2(b) shows the result by using background subtraction. The eight objects shown in Fig. 2(c)can be merged to become three cars, as shown in Fig.2(d).
The procedure of merging headlights is described bellow:
For m=1 toMt, , where Mtis the total numbers of objects at the tthframe.
For n=m+1 toMt
o=0;
If ,whereand are the horizontaland vertical distance of the two centroid coordinateC(m,t) and C(n, t),respectively.
o=o+1;
;(1)
;(2)
End
End
The total object numbers of tth frame is, The centroid coordinate of the oth object at the tth frame is defined as.
The procdure of merging reflected light and headlights is described bellow:
If
For o= 1 to
If, whereand are the horizontal and vertical distance of the two centroid coordinate and C(m, t), respectively.
;(3)
;(4)
Else
=+1;
;(5)
;(6)
End
Once headlights and reflective lights of a car are grouped as a connected component, the object counting procedure at day[19] can be used to find the traffic amount at night.
(a) (b)
(c) (d)
Fig. 2.17th frame of “traffic” video sequence. (a) Originalframe. (b) Result by using background subtraction. (c) Extracted object. (d) Extracted region in the original frame.
(a) (b)
(c) (d)
(e) (f)
Fig. 3. Labeled objects and their trajectoriesof “traffic” video sequence from 171th to176th frames. (a) 171thframe. (b) 172thframe. (c) 173thframe. (d) 174thframe. (e) 175thframe. (f) 176thframe.
Experimental results
In the following experiment, the video sequence is used to evaluate detection at the object level (detecting whether an object is present or not). If a contiguous region of pixels was consistently detected corresponding to an object during its period within the field of view, a correct “object” detection was recorded. If two separate regions were assigned to an object, or if a region was spuriously detected, a misdetection was recorded. Fig. 4 shows the misdetection (Mis-Det.) of object-level for “traffic” video sequence. Fig. 4(a), 4(b) , 4(c) , 4(d),4(e)and 4(f) are the results by using the KDE, MoG, improved MoG, optical flow, BGE and proposed method, respectively. Results, shown in Table 1, demonstrate that the proposed approach had the best performance than the conventional ones.
Fig. 4. Misdetection (Mis-Det.) of object-level for “traffic” video sequence. (a) Result by the KDE method. (b) Result by the MoG method. (c) Result by the improved MoG method. (d) Result by the optical flow method. (e) Result by the BGE method. (f) Result by the proposed method.
Table 1 Accuracy rates of the proposed and conventional segmentation methods.
Methods / KDE / MoG / GMM / Optical flow / BGE / ProposedAccuracy rates / 0.73 / 0.77 / 0.74 / 0.76 / 0.79 / 0.91
Summary
In this work, an automatic method is developed for moving object extraction in video sequences. First, the modified iterative threshold selection technique is adopted to generate the stationary background in which the inconsistent pixels are corrected. Second, this background subtracts the original image to yield a difference image. Third, the robust edge-following method is employed to connect the discontinuous contours of objects resulted from low contrast. Finally, the empty areas of the closed contours are filled by the morphological operation to obtain the complete objects. Particularly at night,two headlights of a car are merged with their corresponding reflective lights on the ground to yield a connected component for identifying a car.Finally, motion connection is effectively applied to track object paths and compute the number of moving objects.The proposed method has been successfully demonstrated by using freeway video sequence at night. The experimental results demonstrate that the proposed method is superior to the conventional methods. Therefore, the method proposed herein can reliably estimate traffic amounts for surveillance applications.
References
[1].A.Yilmaz, O. Javed and M. Shah, “Object tracking: asurvey,”ACM Computing Surveys, vol. 38, no. 4, article 13, Dec. 2006.
[2].B. Sugandi, H. Kim, J. K. Tan and S. Ishikawa,“Real Time Tracking and Identification of Moving Persons by Using a Camera in Outdoor Environment,” International Journal of Innovative Computing, Information and Control, vol. 5, no. 5, pp.1179-1188, May 2009.
[3].C. R. Wren, A. Azarbayejani, T. Darrell and A. P. Pentland, “Pfinder: real-time tracking of the human body,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 780–785, 1997.
[4].L. Maddalena and A. Petrosino, “A self-organizing approach to background subtraction for visual surveillance applications,” IEEE Trans. on Image Processing, vol. 17, no. 7, pp. 1168-1177, July 2008.
[5].M. Irani and P.Anandan, “Video indexing based on mosaic representations,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol.20, no.6, pp.577–589, 1998.
[6].H.Stern and B.Efros, “Adaptive color space switching for face tracking in multi-colored lighting Environments,” Proc. of IEEE International Conference on Automatic Face and Gesture Recognition, pp.249 – 254, May 2002.
[7].A, Cavallaro, O. Steiger and T. Ebrahimi, “Tracking video objects in cluttered background,” IEEE Trans. on Circuits and Systems for Video Technology, vol. 15, no. 4, pp. 575-584, April 2005.
[8].C. Stauffer and W. Grimson, “Learning patterns of activity using real-time tracking,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 747–757, 2000.
[9].Z. Zivkovic and F. Heijden, “Efficient Adaptive Density Estimation per Image Pixel for the Task of Background Subtraction,” Pattern Recognition Letters, vol. 27, no. 7, pp. 773-780, May 2006.
[10].A. Elgammal, R. Duraiswami, D. Harwood and L.S. Davis, “Background and foreground modeling using nonparametric kernel density estimation for visual surveillance,” Proceedings of IEEE, vol. 90, no. 7,pp. 1151-1163, July 2002.
[11].B.Horn, and B. Schunk, “Determining optical flow, “Artific. Intell. vol. 17, pp. 185–203, 1981.
[12].B. D. Lucas, and T. Kanade, “An iterative image registration technique with an application to stereo vision,” International Joint Conference on Artificial Intelligence, pp. 674-679, 1981.
[13].Z. Pan and C.-W. Ngo, “Moving-object detection, association, and selection in home videos,” IEEE Trans. Multimedia, vol. 9, no. 2, pp. 268–279, Feb. 2007.
[14].L. A. Machowski and T. Marwala, “Using an Object Oriented Calculation Process Framework and Neural Networks for Classification of Image Shapes”,International Journal of Innovative Computing, Information and Control, vol. 1, no. 4, pp.609-623, Dec. 2005.
[15].G.F.Franklin, J.D. Powell and M.L. Workman, Digital Control of Dynamic Systems, Second Edition, New York: Addison-Wesley, 1990.
[16].M. Isard and A. Blake, “Condensation—conditional density propagation for visual tracking,”International Journal of Computer Vision, vol.29, no.1, pp.5-28, 1998.
[17].H. Wang, D. Suter, K. Schindler and C. Shen, “Adaptive object tracking based on an effective appearance filter,” IEEE Trans. onPattern Analysis and Machine Intelligence, vol. 29 ,no. 9, pp. 1661- 1667, Sept. 2007.
[18].Y. B. Chen and O. T.-C. Chen,“ High-accuracy moving object extraction using background subtraction,”ICIC Express Letters,vol. 3, no. 4(A), pp. 933-938, Dec.2009.
[19].Y. B. Chen,“Reliable Moving Object Extraction and Counting,” The IET International Conference on Frontier Computing–Theory, Technologies and Applications2010, pp. 432-435, Aug. .2010.