关于司机疲劳监测的人眼检测与跟踪研究
|
摘要
目前交通事故中相当一部分是由于司机疲劳驾驶所造成的，为减了少此类事故的发生，本文研究通过司机的眼睛状态来判断司机是否处于疲劳状态，重点对司机眼睛的检测与跟踪的算法进行了研究与改进。
本文主要作了以下几个方面的工作：
1．为了提高人眼检测的准确率和速度，本文提出了两种人眼检测方法：基于肤色模型和级联增强分类器法和基于几何特征和级联增强分类器法。前一种方法主要用于彩色图像中人眼的检测，后一种方法主要适用于灰度图像，这两种方法对于人眼的检测效果较以前的算法都有所改善。
2．检测出人眼之后就要进行眼睛的跟踪，本文提出了一种新颖的眼睛跟踪方法：将Kalman滤波和Mean Shift算法相结合的方法。整个跟踪过程分为两个阶段：首先根据上一帧图像中眼睛的位置运用Kalman滤波预测当前图像中眼睛的位置和协方差；然后根据眼睛的亮度分布特征运用Mean Shift迭代算法在估计的邻域内搜索与眼睛模板最相似的目标。此眼睛跟踪方法在实际的光照条件下跟踪效果较好，对于眼睛部分遮挡或者闭合的情况也能够准确地跟踪。
3．构建了眼睛跟踪系统，实现了司机眼睛的实时跟踪。
At present some of traffic accidents have taken place because drivers have been in fatigue. In this thesis driver's fatigue level has been estimated according to his eyes state in order to reduce these traffic accidents. Eye detection and tracking methods of the driver have been studied and improved importantly.
In this thesis three facets work have been finished:
Firstly, two methods have been proposed in this thesis that have been used to detect and track eyes, and which are superior to former methods in detecting speed and veracity, one of which is based on color-model and boosted cascade of classifies , and the other is based on geometry feature and boosted cascade of classifies. The former is for color images and the latter is for gray images, and both them are superior to former methods in some degree.
Secondly, novel eye tracking method has been proposed that has conjoined kalman filter and mean shift , the whole tracking process has been divided into two stages: kalman filter is used to estimate eye position and mean shift is used to correct eye position. This eye tracking method is effective for realistic lighting condition, eye part closure and eye closure.
Thirdly, eye tracking system has been constructed and real time eye tracking has been realized.
本文主要作了以下几个方面的工作：
1．为了提高人眼检测的准确率和速度，本文提出了两种人眼检测方法：基于肤色模型和级联增强分类器法和基于几何特征和级联增强分类器法。前一种方法主要用于彩色图像中人眼的检测，后一种方法主要适用于灰度图像，这两种方法对于人眼的检测效果较以前的算法都有所改善。
2．检测出人眼之后就要进行眼睛的跟踪，本文提出了一种新颖的眼睛跟踪方法：将Kalman滤波和Mean Shift算法相结合的方法。整个跟踪过程分为两个阶段：首先根据上一帧图像中眼睛的位置运用Kalman滤波预测当前图像中眼睛的位置和协方差；然后根据眼睛的亮度分布特征运用Mean Shift迭代算法在估计的邻域内搜索与眼睛模板最相似的目标。此眼睛跟踪方法在实际的光照条件下跟踪效果较好，对于眼睛部分遮挡或者闭合的情况也能够准确地跟踪。
3．构建了眼睛跟踪系统，实现了司机眼睛的实时跟踪。
At present some of traffic accidents have taken place because drivers have been in fatigue. In this thesis driver's fatigue level has been estimated according to his eyes state in order to reduce these traffic accidents. Eye detection and tracking methods of the driver have been studied and improved importantly.
In this thesis three facets work have been finished:
Firstly, two methods have been proposed in this thesis that have been used to detect and track eyes, and which are superior to former methods in detecting speed and veracity, one of which is based on color-model and boosted cascade of classifies , and the other is based on geometry feature and boosted cascade of classifies. The former is for color images and the latter is for gray images, and both them are superior to former methods in some degree.
Secondly, novel eye tracking method has been proposed that has conjoined kalman filter and mean shift , the whole tracking process has been divided into two stages: kalman filter is used to estimate eye position and mean shift is used to correct eye position. This eye tracking method is effective for realistic lighting condition, eye part closure and eye closure.
Thirdly, eye tracking system has been constructed and real time eye tracking has been realized.
引文
[1]Transport Research Laboratory. Driver fatigue—a killer on the roads. http://www.ntsb.gov/itsal_private.htm. 1998.10
[2]DrivAlert Systems Inc. Fatigue facts.http://www.zzzzalert.com, 1999.9
[3]Laurence II, Nick M. Review of fatigue detection and prediction technologies.http://www.nrtc.gov.au,2001.1
[4]Allen, R.W., Stein, A.C. & Miller, J.C. (1990). Performance testing as a determinant of fitness-for-duty. Paper 457. Presented at SAE Aerotech'90, Long Beach, California, USA.
[5]Akerstedt, T. & Folkard, S. (1997). The three-process model of alertness and its extension to performance sleep latency and sleep length. Photobiology International, 14(2), pp115-123.
[6]Siegmund, G. P., King, D. J., & Mumford, D. K., Correlation of steering behavior with heavy-truck driver fatigue. Society for Automotive Engineers Technical Report, 1996.
[7]Knipling, R.R. & Wierwille, W.W. (1994). Vehicle-based drowsy driver detection: Current status and future prospects. Proceedings of the IVHS AMERICA Conference. Moving Toward Deployment. Atlanta, Georgia.
[8]Khardi, S. & Vallet, M., "Drowsiness of the driver: EEG and vehicle parameters interaction". Proceedings from the 14'International Technical Conference on the Enhanced Safety of Vehicles, 1994, Paper No, 94-S3-O-06.
[9]Dinges, D.F., Mallis, M.M., Maislin, G. & Powell, J.W. (1998). Final Report: Evaluation of Techniques for Ocular Measurement as an Index of Fatigue and as the Basis for Alertness Management. National Highway Traffic Safety Administration. Report No DOT HS 808 762.
[10]Stern, J. A., Boyer, D., & Schroeder, D. J., "Blink rate as a measure of fatigue: A review". Office of Aviation Medicine, Federal Aviation Administration, Washington, D.C., Report No. DOT/FAA/AM-9407, 1994.
[11]Anon. (1999) Conference on Ocular Measures of Driver Alertness, Washington, DC, April, 1999.
[12]郭俊杰．汽车司机心理与行车安全的关系．道路交通管理，1989，10：p33～37
[13]FHWA report number: FHWA-MC-97-001. http://fhwa.dot.gov,1997.10
[14]Nikolaos P. Vision-based detection of driver fatigue. IEEE International Conference on
Intelligent Transportation.http://www.fmcsn.dot.gov/1997.9
[15]周鹏．疲劳事故隐患消除技术与方法．汽车电器，1998(8)：27～30
[16]Ming-Hsuan, Yang,Narendra and David Kriegman, A Survey on Face Detection Methods, 1999.3.6
[17]P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman. Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 1997.
[18]E. Hjelmas and J. Wroldsen. Recognizing Faces from the Eyes Only. In Proceedings of the 11th Scandinavian Conference on Image Analysis, 1999.
[19]Brunelli, R. and Poggio, T.(1993).Face Recognition: Features Versus Templates, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.15,No.10,pp.1042-1052
[20]容观澳．数字图像处理．北京：清华大学出版社，1999
[21]S. Baluja, and T. Kanade, ~(TM)Neural Network-Based Face Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 20, no. 1, pp. 23-38, Jan. 1998.
[22]边肇祺，张学工等编著．模式识别．北京：清华大学出版社，2001
[23]R.L. Hsu, A. M. Mohamed and A. K. Jain, Face Detection in Color Images, ICIP'2001
[24]陶霖密，彭振云，徐光佑．人体的肤色特征．2000
[25]张宏林编著．数字图像模式识别技术及工程实践．北京：人民邮电出版社，2003
[26]Lam K, Yan H: Locating and extracting the eye in human face images. Pattern Recgnition, 29(5),pp.771-779,1996
[27]Kimme C, Ballard D, Slansky J: Finding circles by an array of accumulators. Communications of the ACM, Vol.18, pp.120-122, 1990
[28]Pohle R, Toennies KD: A three-level evaluation process for segmentation methods in medical imaging, Proceeding of the SPIE, 2002
[29]杨福生．小波变换的工程分析与应用[M]．北京：科学出版社，1999
[30] Yoav Freund, Robert E. Schapire. Experiments with a new boosting algorithm. In Machine Learning: Proceeding of the Thirteenth International Conference, Morgan Kauman, San Francisco, 1996, 148-156.
[31]Y. Freund and R. E. Schapire, A decision-theoretic generalization of on-line learning and an application to boosting, in "Proceedings of the Second European Conference on Computational Learning Theory, 1995," pp. 23-37.
[32] M. Kearns, Thoughts on hypothesis boosting, unpublished, December 1988.
[33]Haralick Shapiro. Computer and Robot Vision, Edi.(1)[M]. Addison Wesley, 1992
[34]Papageorgiou C, Evgeniou T, Poggio T. A trainable pedestrian detection system [J]. Int Intelligent Vehicles, 1998, 10:241 246
[35]Constantine P, Papageorgiou C, Poggio T. A Trainable Object Detection System: Car Detection in Static Images [R]. A. I. Memo 1673, MIT Artificial Intelligence Laboratory, 1999.
[36]Mohan A. Robust Object Detection in Images by Components [D]. Boston: MIT, 1999.
[37]Rainer Lienhart and Jochen Maydt. An Extended Set of Harr-like Features for Rapid Object Detection. IEEE ICIP 2002, Vol.1, pp. 900-903, Sep. 2002
[38]A.Mohan, C. Papageorgiou, T. Poggio, Example-based object detection in images by components. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.23.No.4,pp. 349-361, April 2001
[39]C. Papageorgiou, M.Oren, and T. Poggio. A general frame work for Object Detection. In International Conference on Computer Vision, 1998
[40]Paul Viola and Michael J.Jones. Rapid Object Detection using a Boosted Cascade of Simple Features. IEEE CVPR, 2001
[41]Willliam T. Freeman and Edward H.Adelson. The design and use of steerable filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(9):891-906, 1991
[42]H. Greenspan, S. Belongie, R. Gooodman, P. Perona, S,. Rakshit, and C. Anderson. Overcomplete steerable pyramid filters and rotation invariance. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognitioon, 1994
[43]Robert E. SsCHAPIRE, Yoav Freund, Peter Bartlett, and Wee Sun Lee. Boosting the margin: A new explanation for the effectiveness of voting methods. In Proceeding s of the Fourteenth International Conference on Machine Learning, 1997
[44]Haralick, Shapiro. Computer and Robot[J]. Addision-Wesley, 1992, 1: 59-83, 639-658.
[45]Image Processing Toolbox[M]. In: MATLAB.
[46]Y. Ebisawa. Unconstrained pupil detection technique using tow light sources and the image difference method. Visualization and Intelligent Design in engineering and architecture, pages 79-80, 1995
[47]C.H. Morimoto and M. Flickner. Real-time multiple face detection using active illumination. Proc. Of the 4th IEEE International Conference on Automatic Face and Gesture Recognition 2000,Grenoble, France, March,2000.
[48]A. Haro, M. Flickner, and I. Essa. Detecting and Tracking Eyes By Using Their
Physiological Properties, Dynamics, and Appearance. Proceeding IEEE CVPR 2000, Hilton Head Island, South Carolina, June 2000.
[49]Q. Ji and X. Yang, Real time visual cues extraction for monitoring driver vigilance, In Proc. Of International Workshop on Computer Vision Systems, July 7-8, 2001, Vancouver, Canada.
[50]Paul Smith, Mubarak Shah and N. da Vitoria Lobo. Monitoring head/eye motion for driver alertness with one camera. Proceedings of the International Conference on Pattern Recognition, Session P4.3A, 2000
[51]D. Comaniciu, V. Ramesh, P. Meer, Real-Time Tracking of Non-Rigid Objects using Mean Shift, IEEE Conf. on Comp. Vis. And Pat. Rec., Hilton Head Island, South Carolina, 2000.
[52]D. Comaniciu, V. Ramesh, Mean Shift and Optimal Prediction for Efficient Object Tracking, IEEE Int'l Conf. Image Processing, Vancouver, Canada, Vol.3,70-73,2000
[53]D. Comaniciu, P. Meer, "Mean Shift Analysis and Applications," IEEE Int'l Conf. Comp. Vis., Kerkyra, Greece, 1197-1203, 1999.
[54]A. Djouadi, O. Snorrason, F.D. Garber. The Quality of Training-Sample Estimates of the Bhattacharyya Coefficient. IEEE Trans. Pattern Analysis Machine Intelligent, 1990, 12: 92-97.
[55]T. Kailath. The Divergence and Bhattacharyya Distance Measures in Signal Selection. IEEE Trans. Commun. Tech., 1967, COM-15: 52-60.
[56]Dinges D F, Grace R. PERCLOS:A valid Psychophysiological measure of alertness as assessed by psychomotor vigilance, http://fhwa.dot.gov. 1998-10
[57]Federal Highway Administration. Eye- activity measures of fatigue and napping as a fatigue countermeasure. Federal Highway Administration Tech Brief. http://fhwa.dot.gov. 1994-04
[58]L aurence H, Nick M. Review of fatigue detection and prediction technologies. http://www.nrtc.Gov.au.
[59]何斌等著．Visual C++数字图像处理．北京：人民邮电出版社，2003
[60]潘爱民，王国印译．Visual C++技术内幕．北京：清华大学出版社，2001
[2]DrivAlert Systems Inc. Fatigue facts.http://www.zzzzalert.com, 1999.9
[3]Laurence II, Nick M. Review of fatigue detection and prediction technologies.http://www.nrtc.gov.au,2001.1
[4]Allen, R.W., Stein, A.C. & Miller, J.C. (1990). Performance testing as a determinant of fitness-for-duty. Paper 457. Presented at SAE Aerotech'90, Long Beach, California, USA.
[5]Akerstedt, T. & Folkard, S. (1997). The three-process model of alertness and its extension to performance sleep latency and sleep length. Photobiology International, 14(2), pp115-123.
[6]Siegmund, G. P., King, D. J., & Mumford, D. K., Correlation of steering behavior with heavy-truck driver fatigue. Society for Automotive Engineers Technical Report, 1996.
[7]Knipling, R.R. & Wierwille, W.W. (1994). Vehicle-based drowsy driver detection: Current status and future prospects. Proceedings of the IVHS AMERICA Conference. Moving Toward Deployment. Atlanta, Georgia.
[8]Khardi, S. & Vallet, M., "Drowsiness of the driver: EEG and vehicle parameters interaction". Proceedings from the 14'International Technical Conference on the Enhanced Safety of Vehicles, 1994, Paper No, 94-S3-O-06.
[9]Dinges, D.F., Mallis, M.M., Maislin, G. & Powell, J.W. (1998). Final Report: Evaluation of Techniques for Ocular Measurement as an Index of Fatigue and as the Basis for Alertness Management. National Highway Traffic Safety Administration. Report No DOT HS 808 762.
[10]Stern, J. A., Boyer, D., & Schroeder, D. J., "Blink rate as a measure of fatigue: A review". Office of Aviation Medicine, Federal Aviation Administration, Washington, D.C., Report No. DOT/FAA/AM-9407, 1994.
[11]Anon. (1999) Conference on Ocular Measures of Driver Alertness, Washington, DC, April, 1999.
[12]郭俊杰．汽车司机心理与行车安全的关系．道路交通管理，1989，10：p33～37
[13]FHWA report number: FHWA-MC-97-001. http://fhwa.dot.gov,1997.10
[14]Nikolaos P. Vision-based detection of driver fatigue. IEEE International Conference on
Intelligent Transportation.http://www.fmcsn.dot.gov/1997.9
[15]周鹏．疲劳事故隐患消除技术与方法．汽车电器，1998(8)：27～30
[16]Ming-Hsuan, Yang,Narendra and David Kriegman, A Survey on Face Detection Methods, 1999.3.6
[17]P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman. Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 1997.
[18]E. Hjelmas and J. Wroldsen. Recognizing Faces from the Eyes Only. In Proceedings of the 11th Scandinavian Conference on Image Analysis, 1999.
[19]Brunelli, R. and Poggio, T.(1993).Face Recognition: Features Versus Templates, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.15,No.10,pp.1042-1052
[20]容观澳．数字图像处理．北京：清华大学出版社，1999
[21]S. Baluja, and T. Kanade, ~(TM)Neural Network-Based Face Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 20, no. 1, pp. 23-38, Jan. 1998.
[22]边肇祺，张学工等编著．模式识别．北京：清华大学出版社，2001
[23]R.L. Hsu, A. M. Mohamed and A. K. Jain, Face Detection in Color Images, ICIP'2001
[24]陶霖密，彭振云，徐光佑．人体的肤色特征．2000
[25]张宏林编著．数字图像模式识别技术及工程实践．北京：人民邮电出版社，2003
[26]Lam K, Yan H: Locating and extracting the eye in human face images. Pattern Recgnition, 29(5),pp.771-779,1996
[27]Kimme C, Ballard D, Slansky J: Finding circles by an array of accumulators. Communications of the ACM, Vol.18, pp.120-122, 1990
[28]Pohle R, Toennies KD: A three-level evaluation process for segmentation methods in medical imaging, Proceeding of the SPIE, 2002
[29]杨福生．小波变换的工程分析与应用[M]．北京：科学出版社，1999
[30] Yoav Freund, Robert E. Schapire. Experiments with a new boosting algorithm. In Machine Learning: Proceeding of the Thirteenth International Conference, Morgan Kauman, San Francisco, 1996, 148-156.
[31]Y. Freund and R. E. Schapire, A decision-theoretic generalization of on-line learning and an application to boosting, in "Proceedings of the Second European Conference on Computational Learning Theory, 1995," pp. 23-37.
[32] M. Kearns, Thoughts on hypothesis boosting, unpublished, December 1988.
[33]Haralick Shapiro. Computer and Robot Vision, Edi.(1)[M]. Addison Wesley, 1992
[34]Papageorgiou C, Evgeniou T, Poggio T. A trainable pedestrian detection system [J]. Int Intelligent Vehicles, 1998, 10:241 246
[35]Constantine P, Papageorgiou C, Poggio T. A Trainable Object Detection System: Car Detection in Static Images [R]. A. I. Memo 1673, MIT Artificial Intelligence Laboratory, 1999.
[36]Mohan A. Robust Object Detection in Images by Components [D]. Boston: MIT, 1999.
[37]Rainer Lienhart and Jochen Maydt. An Extended Set of Harr-like Features for Rapid Object Detection. IEEE ICIP 2002, Vol.1, pp. 900-903, Sep. 2002
[38]A.Mohan, C. Papageorgiou, T. Poggio, Example-based object detection in images by components. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.23.No.4,pp. 349-361, April 2001
[39]C. Papageorgiou, M.Oren, and T. Poggio. A general frame work for Object Detection. In International Conference on Computer Vision, 1998
[40]Paul Viola and Michael J.Jones. Rapid Object Detection using a Boosted Cascade of Simple Features. IEEE CVPR, 2001
[41]Willliam T. Freeman and Edward H.Adelson. The design and use of steerable filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(9):891-906, 1991
[42]H. Greenspan, S. Belongie, R. Gooodman, P. Perona, S,. Rakshit, and C. Anderson. Overcomplete steerable pyramid filters and rotation invariance. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognitioon, 1994
[43]Robert E. SsCHAPIRE, Yoav Freund, Peter Bartlett, and Wee Sun Lee. Boosting the margin: A new explanation for the effectiveness of voting methods. In Proceeding s of the Fourteenth International Conference on Machine Learning, 1997
[44]Haralick, Shapiro. Computer and Robot[J]. Addision-Wesley, 1992, 1: 59-83, 639-658.
[45]Image Processing Toolbox[M]. In: MATLAB.
[46]Y. Ebisawa. Unconstrained pupil detection technique using tow light sources and the image difference method. Visualization and Intelligent Design in engineering and architecture, pages 79-80, 1995
[47]C.H. Morimoto and M. Flickner. Real-time multiple face detection using active illumination. Proc. Of the 4th IEEE International Conference on Automatic Face and Gesture Recognition 2000,Grenoble, France, March,2000.
[48]A. Haro, M. Flickner, and I. Essa. Detecting and Tracking Eyes By Using Their
Physiological Properties, Dynamics, and Appearance. Proceeding IEEE CVPR 2000, Hilton Head Island, South Carolina, June 2000.
[49]Q. Ji and X. Yang, Real time visual cues extraction for monitoring driver vigilance, In Proc. Of International Workshop on Computer Vision Systems, July 7-8, 2001, Vancouver, Canada.
[50]Paul Smith, Mubarak Shah and N. da Vitoria Lobo. Monitoring head/eye motion for driver alertness with one camera. Proceedings of the International Conference on Pattern Recognition, Session P4.3A, 2000
[51]D. Comaniciu, V. Ramesh, P. Meer, Real-Time Tracking of Non-Rigid Objects using Mean Shift, IEEE Conf. on Comp. Vis. And Pat. Rec., Hilton Head Island, South Carolina, 2000.
[52]D. Comaniciu, V. Ramesh, Mean Shift and Optimal Prediction for Efficient Object Tracking, IEEE Int'l Conf. Image Processing, Vancouver, Canada, Vol.3,70-73,2000
[53]D. Comaniciu, P. Meer, "Mean Shift Analysis and Applications," IEEE Int'l Conf. Comp. Vis., Kerkyra, Greece, 1197-1203, 1999.
[54]A. Djouadi, O. Snorrason, F.D. Garber. The Quality of Training-Sample Estimates of the Bhattacharyya Coefficient. IEEE Trans. Pattern Analysis Machine Intelligent, 1990, 12: 92-97.
[55]T. Kailath. The Divergence and Bhattacharyya Distance Measures in Signal Selection. IEEE Trans. Commun. Tech., 1967, COM-15: 52-60.
[56]Dinges D F, Grace R. PERCLOS:A valid Psychophysiological measure of alertness as assessed by psychomotor vigilance, http://fhwa.dot.gov. 1998-10
[57]Federal Highway Administration. Eye- activity measures of fatigue and napping as a fatigue countermeasure. Federal Highway Administration Tech Brief. http://fhwa.dot.gov. 1994-04
[58]L aurence H, Nick M. Review of fatigue detection and prediction technologies. http://www.nrtc.Gov.au.
[59]何斌等著．Visual C++数字图像处理．北京：人民邮电出版社，2003
[60]潘爱民，王国印译．Visual C++技术内幕．北京：清华大学出版社，2001