基于摄像头交汇的屏幕多点定位系统的研究与实现
|
摘要
交互式电子设备作为一种先进的教学或会议辅助人机交互产品,凭着其高清大屏幕的集中显示、直观的屏幕手写标注、便捷的触摸操作、海量图像的快速处理、远程多方数据共享以及人性化的操作模式等诸多优势,已广泛应用于商业、企业、教育、科研、政府、军事、医疗等领域的会议室、演示厅、教室、指挥室等等,为满足群体及时、互动沟通与现场决策提供了全方位的解决方案。
多点触控技术正是交互式电子设备解决方案中的一项关键技术。多点触控技术包含三个关键的技术层面,一是多点精确定位,二是多点跟踪识别处理,三是手势识别处理。而多点精确定位是实现多点触控技术的基础,它在多点触控系统中起着非常关键的作用。
目前,有多种技术可以实现交互式电子设备的精确定位,具有代表性的是电磁感应、红外线、电阻膜、超声波、CCD(Charge-Coupled Device电荷耦合元件)光扫描等技术。目前基于这些技术研制的定位技术大多存在技术复杂度高、成本高昂、应用范围狭窄、难以实现多点定位等缺点。
本文基于光学检测的一系列理论和数码摄像技术的相关知识,充分借鉴CCD光扫描定位原理,在IDB4665(Interactive Digital Board交互数字平台)实验平台上,采用低成本的CMOS(Complementary Metal Oxide Semiconductor互补金属氧化物半导体)数字图像传感器作为图像采集器件,利用摄像头交汇定位原理和图像畸变校正技术,实现了屏幕表面的精确多点定位,为交互式系统的多点定位提供了新的解决方案,具有较高的应用价值。
本文研究的重点和难点包括基于摄像头交汇的屏幕多点定位系统的体系结构的构建、摄像头交汇定位的设计和实现以及广角镜头、鱼眼镜头畸变图像精确校正的算法选择和实现等内容。
Interactive electronic device is an advanced teaching and meeting products supporting human-computer interaction. It has big high-definition screen, intuitive on-screen handwriting annotation, easy touch operation, mass rapid image processing, remote multi-party data sharing and user-friendly mode of operation and many other advantages. It has been widely used in meeting rooms, classrooms, demonstration rooms, control rooms of commercial, business, education, scientific research, military, government, and health care, and it provides a comprehensive solution for people’s quickly interactive communication and on-site decision-making.
Multi-Touch is an important technology in solutions of interactive electronic equipments. It is composed of three important technologies, one is precise-positioning, the second is tracking-recognition, the third is gesture-recognition. The multi-point precise-positioning is the base of multi-touch technology, it plays a very important role in multi-touch system.
Currently, there are some technologies can achieve precise-positioning of interactive electronic device, Including the electromagnetic induction, infrared ray, resistive membrane, ultrasonic wave, CCD (Charge-Coupled Device Charge-coupled device) light scanning technology, and so on. Most of the technologies are complexity, high-cost, limited range of applications, and they are difficult to implement.
This paper based on a series of theory of optical detection , the knowledge of digital camera and the technology of CCD light-scanning. We use the low-cost CMOS (Complementary Metal Oxide Semiconductor) digital image sensor as the image acquisition device. Based on IDB4665 (Interactive Digital Board) test platform, we achieve the precise positioning system by using the technology of image distortion correction and convergence of camera location, which indicates that a new approach with application value is issued for the realization of people's multi-point positioning system.
There are several important and difficult technology in the paper. The construction of architecture of system for multi-point positioning of screen based on camera convergence. The design and implementation of camera positioning, and the selection and implementation of distortion correction algorithm for wide-angle lens, fisheye lens.
多点触控技术正是交互式电子设备解决方案中的一项关键技术。多点触控技术包含三个关键的技术层面,一是多点精确定位,二是多点跟踪识别处理,三是手势识别处理。而多点精确定位是实现多点触控技术的基础,它在多点触控系统中起着非常关键的作用。
目前,有多种技术可以实现交互式电子设备的精确定位,具有代表性的是电磁感应、红外线、电阻膜、超声波、CCD(Charge-Coupled Device电荷耦合元件)光扫描等技术。目前基于这些技术研制的定位技术大多存在技术复杂度高、成本高昂、应用范围狭窄、难以实现多点定位等缺点。
本文基于光学检测的一系列理论和数码摄像技术的相关知识,充分借鉴CCD光扫描定位原理,在IDB4665(Interactive Digital Board交互数字平台)实验平台上,采用低成本的CMOS(Complementary Metal Oxide Semiconductor互补金属氧化物半导体)数字图像传感器作为图像采集器件,利用摄像头交汇定位原理和图像畸变校正技术,实现了屏幕表面的精确多点定位,为交互式系统的多点定位提供了新的解决方案,具有较高的应用价值。
本文研究的重点和难点包括基于摄像头交汇的屏幕多点定位系统的体系结构的构建、摄像头交汇定位的设计和实现以及广角镜头、鱼眼镜头畸变图像精确校正的算法选择和实现等内容。
Interactive electronic device is an advanced teaching and meeting products supporting human-computer interaction. It has big high-definition screen, intuitive on-screen handwriting annotation, easy touch operation, mass rapid image processing, remote multi-party data sharing and user-friendly mode of operation and many other advantages. It has been widely used in meeting rooms, classrooms, demonstration rooms, control rooms of commercial, business, education, scientific research, military, government, and health care, and it provides a comprehensive solution for people’s quickly interactive communication and on-site decision-making.
Multi-Touch is an important technology in solutions of interactive electronic equipments. It is composed of three important technologies, one is precise-positioning, the second is tracking-recognition, the third is gesture-recognition. The multi-point precise-positioning is the base of multi-touch technology, it plays a very important role in multi-touch system.
Currently, there are some technologies can achieve precise-positioning of interactive electronic device, Including the electromagnetic induction, infrared ray, resistive membrane, ultrasonic wave, CCD (Charge-Coupled Device Charge-coupled device) light scanning technology, and so on. Most of the technologies are complexity, high-cost, limited range of applications, and they are difficult to implement.
This paper based on a series of theory of optical detection , the knowledge of digital camera and the technology of CCD light-scanning. We use the low-cost CMOS (Complementary Metal Oxide Semiconductor) digital image sensor as the image acquisition device. Based on IDB4665 (Interactive Digital Board) test platform, we achieve the precise positioning system by using the technology of image distortion correction and convergence of camera location, which indicates that a new approach with application value is issued for the realization of people's multi-point positioning system.
There are several important and difficult technology in the paper. The construction of architecture of system for multi-point positioning of screen based on camera convergence. The design and implementation of camera positioning, and the selection and implementation of distortion correction algorithm for wide-angle lens, fisheye lens.
引文
[1]广东威创视讯科技股份有限公司.交互数字平台[EB/OL]. http://www.vtron.com/IDB/pro_intro.jsp?catid=129|180|181|183
[2] 3see市场研究信息网.2009年中国交互式电子白板发展状况调查报告[R].北京:北京信通四方企业顾问有限公司,2010
[3] Microsoft.What is Microsoft Surface?[EB/OL].http://www.microsoft.com/surface/
[4] Han J. Y. Low-Cost Multi-Touch Sensing through Frustrated Total Internal Reflection. In Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology, pp. 115-118. 2005.
[5] SMART Technologies Inc. Digital-Vision Touch (DViTTM) Technology white paper. February 2003. URL: http://www.smarttech.com/DViT/
[6] S. Bougnoux,“From Projective to Euclidean Space under any Practical Situation, a Criticism of Self-Calibration”, Proc. Sixth Int'l Conf. Computer Vision, pp. 790-796, Jan. 1998.
[7] D.C. Brown,“Close-Range Camera Calibration”, Photogrammetric Eng., vol. 37, no. 8, pp. 855-866, 1971.
[8] W. Faig,“Calibration of Close-Range Photogrammetry Systems: Mathematical Formulation”, Photogrammetric Eng. and Remote Sensing, vol. 41, no. 12, pp. 1,479-1,486, 1975.
[9]卢如西,电子显示系统的多点定位装置及方法[P],中国:ACF2090611CN,2006
[10] O. Faugeras, Three-Dimensional Computer Vision: A Geometric Viewpoint. MIT Press, 1993.
[11] O. Faugeras, T. Luong, and S. Maybank,“Camera Self-Calibration: Theory and Experiments”, Proc Second European Conf. Computer Vision, pp. 321-334, May 1992.
[12] O. Faugeras and G. Toscani,“The Calibration Problem for Stereo”, Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 15-20, June 1986.
[13]张全法,何金田.提高广角成像系统几何畸变数字校正精度的方法[J ] .光学技术,2001 ,27 (3) :242-246.
[14]刘航,郁道银,杜吉,等.广角成像系统光学畸变的数字校正方法[J ] .光学学报,1998,18 (8) :1108-1112.
[15] Weng Juyang, PaulCohen, MarcHernious. Camera calibration with distortion models and accuracy evaluation. IEEE Transaction on Pattern Analysis and Machine Intelligence,1992; 14(10): 965-980
[16]阮秋琦,数字图像处理学,北京:电子工业出版社, 2007
[17]马颂德,张正友,计算机视觉—计算理论与算法基础[M].北京:科学出版社, 1998: 72-78.
[18] Abdel-Aziz Y I, Karara H M. Direct linear transformation into object space coordinates in close-range photogram-metry. In:Proc. Symp. Close-Range Photogrammetry. 1971. 1~18
[19] Brown D C. Decentering Distortion of Lenses.Photogrammetric Eng. Remote Sensing. 1966.444~462
[20] Wong K W. Mathematical foundation and digital analysis in close-range photogrammetry. In:Proc.13th Congress of the Int. Society for Photogrammetry. 1976,1355~1373
[21] Tsai R. Y, A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses[J]. IEEE Journal of Robotics Automation,1987,3(4):323-344.
[22]杨必武,郭晓松,摄像机镜头非线性畸变校正方法综述[J].中国图象图形学报,2005,10(3): 269-274.
[23] LIU C,YU Z,CHE R,et al. Bundle adjustment CCD camera calibration based on collinearity equation [J]. Chinese Journal of Mechanical Engineering,2004,17(4):494-497.
[24]周富强,胡坤,张广军.基于共线特征点的摄像机镜头畸变校正[J].机械工程学报,2006,9(42):174-177.
[25] MOUMEN A,ALY F. Nonmetric calibration of camera lens distortion:Differential methods and robust estimation [J]. IEEE Transaction on Image Processing,2005,14(8):1215-1230.
[26] PAN F,WANG X. New versatile camera calibration technique based on linear rectification [J]. Chinese Journal of Mechanical Engineering,2004,17(4):507-510.
[27]朱铮涛,黎绍发,陈华平,基于平面网格模型的摄像机镜头畸变校正技术[ J].计算机工程与应用,2004,23(3): 32-35
[28]卢泉,刘上乾,王会峰,基于同心圆现场标定的实时畸变校正方法[J].光电工程,2009,36(10):81-85
[29]苏小华,等.CCD摄像机成像畸变的研究[J].物理实验,23(9):39-41.
[30]廖士中,高培焕,等.一种光学镜头摄像机图象几何畸变的修正方法[J].中国图象图形学报,2000,5(A)(7):593-596.
[31]姜大志,郁倩,王冰洋,等.计算机视觉成像的非线性畸变研究与综述[J].计算机工程,2001,27(12):108~110.
[32] Feng Wenyi,Liu Bin, LingXieting,etal. Unsupervised image shape restoration based on IDP[J]. Journal of Fudan University(Natural Science), 1995,34(3): 255-261.
[33]史泽林,康娇,孙锐.基于BP神经网络的大视场成像畸变校正方法[J].光学精密工程,2005,13(3):348-353.
[34]张全法.平面物体在曲面状态下扫描仪图像的校正实验[J].光学技术,2003,29(4):454-457.
[35]王欣刚.红外观察仪图像畸变的校正[J].光学精密工程,2001,9(1):39-42.
[36]闵乐泉.一种恢复数字化畸变凸性图像的算法[J].北京科技大学学报,2001,23(6) : 573-576.
[37] Tsai R. Y, An efficient and accurate camera calibration technique for 3D maehine vision[A] .In:Proceeding of International Conference on Computer Vision and Pattern Recognition[Cl,Miami Beaeh,FL,USA,1986,6:364-374.
[38]王亚东,丁明跃,彭嘉雄,一种基于摄像机模型的畸变图像校正方法[J].自动化学报,1997,23( 5 ):717~720.
[39]张艳珍,欧宗瑛,薛斌党.一种基于斜率的摄像机畸变校正方法[J].小型微型计算机系统,2002,23(5):625-627
[40]崔洪州,孔渊,周起勃,等.基于畸变率的图像几何校正[J].应用光学,2006,27(3):183-185
[41]韩广良,宋建中.一种基于畸变等效曲面的图像畸变校正[J].光学技术,2005,31(1):122-124.
[42]郭羽,杨红,杨照金,等.CCD摄像系统镜头的畸变测量[J].应用光学,2008,29(2):279-282.
[43]王永仲,鱼眼镜头光学[M].北京:科学出版社,2006
[44]杨广全,鱼眼图像的校正算法[J].光机电信息,2009,26(4):43-46.
[2] 3see市场研究信息网.2009年中国交互式电子白板发展状况调查报告[R].北京:北京信通四方企业顾问有限公司,2010
[3] Microsoft.What is Microsoft Surface?[EB/OL].http://www.microsoft.com/surface/
[4] Han J. Y. Low-Cost Multi-Touch Sensing through Frustrated Total Internal Reflection. In Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology, pp. 115-118. 2005.
[5] SMART Technologies Inc. Digital-Vision Touch (DViTTM) Technology white paper. February 2003. URL: http://www.smarttech.com/DViT/
[6] S. Bougnoux,“From Projective to Euclidean Space under any Practical Situation, a Criticism of Self-Calibration”, Proc. Sixth Int'l Conf. Computer Vision, pp. 790-796, Jan. 1998.
[7] D.C. Brown,“Close-Range Camera Calibration”, Photogrammetric Eng., vol. 37, no. 8, pp. 855-866, 1971.
[8] W. Faig,“Calibration of Close-Range Photogrammetry Systems: Mathematical Formulation”, Photogrammetric Eng. and Remote Sensing, vol. 41, no. 12, pp. 1,479-1,486, 1975.
[9]卢如西,电子显示系统的多点定位装置及方法[P],中国:ACF2090611CN,2006
[10] O. Faugeras, Three-Dimensional Computer Vision: A Geometric Viewpoint. MIT Press, 1993.
[11] O. Faugeras, T. Luong, and S. Maybank,“Camera Self-Calibration: Theory and Experiments”, Proc Second European Conf. Computer Vision, pp. 321-334, May 1992.
[12] O. Faugeras and G. Toscani,“The Calibration Problem for Stereo”, Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 15-20, June 1986.
[13]张全法,何金田.提高广角成像系统几何畸变数字校正精度的方法[J ] .光学技术,2001 ,27 (3) :242-246.
[14]刘航,郁道银,杜吉,等.广角成像系统光学畸变的数字校正方法[J ] .光学学报,1998,18 (8) :1108-1112.
[15] Weng Juyang, PaulCohen, MarcHernious. Camera calibration with distortion models and accuracy evaluation. IEEE Transaction on Pattern Analysis and Machine Intelligence,1992; 14(10): 965-980
[16]阮秋琦,数字图像处理学,北京:电子工业出版社, 2007
[17]马颂德,张正友,计算机视觉—计算理论与算法基础[M].北京:科学出版社, 1998: 72-78.
[18] Abdel-Aziz Y I, Karara H M. Direct linear transformation into object space coordinates in close-range photogram-metry. In:Proc. Symp. Close-Range Photogrammetry. 1971. 1~18
[19] Brown D C. Decentering Distortion of Lenses.Photogrammetric Eng. Remote Sensing. 1966.444~462
[20] Wong K W. Mathematical foundation and digital analysis in close-range photogrammetry. In:Proc.13th Congress of the Int. Society for Photogrammetry. 1976,1355~1373
[21] Tsai R. Y, A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses[J]. IEEE Journal of Robotics Automation,1987,3(4):323-344.
[22]杨必武,郭晓松,摄像机镜头非线性畸变校正方法综述[J].中国图象图形学报,2005,10(3): 269-274.
[23] LIU C,YU Z,CHE R,et al. Bundle adjustment CCD camera calibration based on collinearity equation [J]. Chinese Journal of Mechanical Engineering,2004,17(4):494-497.
[24]周富强,胡坤,张广军.基于共线特征点的摄像机镜头畸变校正[J].机械工程学报,2006,9(42):174-177.
[25] MOUMEN A,ALY F. Nonmetric calibration of camera lens distortion:Differential methods and robust estimation [J]. IEEE Transaction on Image Processing,2005,14(8):1215-1230.
[26] PAN F,WANG X. New versatile camera calibration technique based on linear rectification [J]. Chinese Journal of Mechanical Engineering,2004,17(4):507-510.
[27]朱铮涛,黎绍发,陈华平,基于平面网格模型的摄像机镜头畸变校正技术[ J].计算机工程与应用,2004,23(3): 32-35
[28]卢泉,刘上乾,王会峰,基于同心圆现场标定的实时畸变校正方法[J].光电工程,2009,36(10):81-85
[29]苏小华,等.CCD摄像机成像畸变的研究[J].物理实验,23(9):39-41.
[30]廖士中,高培焕,等.一种光学镜头摄像机图象几何畸变的修正方法[J].中国图象图形学报,2000,5(A)(7):593-596.
[31]姜大志,郁倩,王冰洋,等.计算机视觉成像的非线性畸变研究与综述[J].计算机工程,2001,27(12):108~110.
[32] Feng Wenyi,Liu Bin, LingXieting,etal. Unsupervised image shape restoration based on IDP[J]. Journal of Fudan University(Natural Science), 1995,34(3): 255-261.
[33]史泽林,康娇,孙锐.基于BP神经网络的大视场成像畸变校正方法[J].光学精密工程,2005,13(3):348-353.
[34]张全法.平面物体在曲面状态下扫描仪图像的校正实验[J].光学技术,2003,29(4):454-457.
[35]王欣刚.红外观察仪图像畸变的校正[J].光学精密工程,2001,9(1):39-42.
[36]闵乐泉.一种恢复数字化畸变凸性图像的算法[J].北京科技大学学报,2001,23(6) : 573-576.
[37] Tsai R. Y, An efficient and accurate camera calibration technique for 3D maehine vision[A] .In:Proceeding of International Conference on Computer Vision and Pattern Recognition[Cl,Miami Beaeh,FL,USA,1986,6:364-374.
[38]王亚东,丁明跃,彭嘉雄,一种基于摄像机模型的畸变图像校正方法[J].自动化学报,1997,23( 5 ):717~720.
[39]张艳珍,欧宗瑛,薛斌党.一种基于斜率的摄像机畸变校正方法[J].小型微型计算机系统,2002,23(5):625-627
[40]崔洪州,孔渊,周起勃,等.基于畸变率的图像几何校正[J].应用光学,2006,27(3):183-185
[41]韩广良,宋建中.一种基于畸变等效曲面的图像畸变校正[J].光学技术,2005,31(1):122-124.
[42]郭羽,杨红,杨照金,等.CCD摄像系统镜头的畸变测量[J].应用光学,2008,29(2):279-282.
[43]王永仲,鱼眼镜头光学[M].北京:科学出版社,2006
[44]杨广全,鱼眼图像的校正算法[J].光机电信息,2009,26(4):43-46.