面向数字城市旅游的视频建模关键技术的研究与开发
|
摘要
随着互联网的发展,虚拟旅游近年来成为了互联网发展的一个热点。虚拟旅游利用越来越成熟的视频技术和电子地图技术,把两者有机结合起来,使用户可以不出门便可得到身临其境的感觉。同时,这种方法也是现代化的城市所需要的现代化宣传手段。现在通过网络的虚拟游览技术向世界展示城市的迷人风貌,吸引旅游和投资者,已经成为热点。同时,把虚拟旅游技术和传统的三维建模技术结合起来,便可把这些技术应用到电子商务中去。人们在虚拟漫游一个城市的时候,同时可以浏览和购买通过三维建模得到的商品的三维模型。这也是对传统技术的一个应用创新。但是,目前的虚拟旅游大多数都是基于普通视频或者是全景图片,这两种方式都缺乏感染力,使用户缺乏沉浸感。同时,在三维重建中,图象匹配技术是一项基本的且很重要的技术。现在的一些算法在输入的初始匹配的outliers比较多的时候,产生的结果中包含很多不匹配的点,因此,这些方法都值得改进。
基于此,在本文中,首先,我们提出了两种对现有的基于随机采样的,应用于图象特征点匹配的LMedS算法的改进。这两个改进后的算法在输入的初始匹配点的outliers比例比较高的时候,能够产生很好的效果。我们通过构造各种实验数据证明,我们的算法的实验结果比传统的LMedS算法的实验结果好很多,因此我们的改进是很有效果的。其次,本文展示了我们开发的一个基于电子地图和全景视频的旅游展示平台。这个平台包括了一个基于网页的Flash全景视频播放器。并且,这个播放器的视频流是和电子地图的定位功能结合起来,由电子地图的定位功能来指导视频流的播放位置,同时,视频流的播放位置也改变电子地图的当前位置。
Recent years, with the rapid development of the Internet, the Virtual Touring is becoming more and more attractive. This technology combines the video technology and the e-map and can make the users feel as if they were touring in a city without going to this city. At the same time, this technology is widely used in the contribution of a city. The people can make a video of the city and put it on the internet, and let the whole world know it. This method is a new way of attracting travelers. And the Virtual Touring technology and the traditional 3D reconstruction technology can be combined in the e-commence area. When the traveler is virtual touring a city, he can go into a shop and view the goods constructed by the 3D reconstruction technology. This is a new application of the traditional technology. At present, most applications of Virtual Touring use normal videos or omni-directional videos. These methods don't have much influence on the viewer. And at present, many algorithms in the feature matching technology, which is a fundamental technology in 3D reconstruction, worth improving when the input matching has many outliers in it.
Based on this, firstly, this paper will introduce two improvements to the LMedS algorithm, which is a random sampling based algorithm used in pattern recognition and feature matching. The two new algorithms will improve the LMedS algorithm when the percentage of outliers is high. By constructing data and feed these data to the algorithms, we show that the new two algorithm swill improve the old methods. And then, we show a platform which combines the e-map and the omni-directional video. This includes an IE based omni-directional video player, which is developed by Flash. The platform combines the e-map and the omni-directional in such a way that the video stream from which the play starts to play is given by the e-map and the e-map also shows the current position of the video stream.
基于此,在本文中,首先,我们提出了两种对现有的基于随机采样的,应用于图象特征点匹配的LMedS算法的改进。这两个改进后的算法在输入的初始匹配点的outliers比例比较高的时候,能够产生很好的效果。我们通过构造各种实验数据证明,我们的算法的实验结果比传统的LMedS算法的实验结果好很多,因此我们的改进是很有效果的。其次,本文展示了我们开发的一个基于电子地图和全景视频的旅游展示平台。这个平台包括了一个基于网页的Flash全景视频播放器。并且,这个播放器的视频流是和电子地图的定位功能结合起来,由电子地图的定位功能来指导视频流的播放位置,同时,视频流的播放位置也改变电子地图的当前位置。
Recent years, with the rapid development of the Internet, the Virtual Touring is becoming more and more attractive. This technology combines the video technology and the e-map and can make the users feel as if they were touring in a city without going to this city. At the same time, this technology is widely used in the contribution of a city. The people can make a video of the city and put it on the internet, and let the whole world know it. This method is a new way of attracting travelers. And the Virtual Touring technology and the traditional 3D reconstruction technology can be combined in the e-commence area. When the traveler is virtual touring a city, he can go into a shop and view the goods constructed by the 3D reconstruction technology. This is a new application of the traditional technology. At present, most applications of Virtual Touring use normal videos or omni-directional videos. These methods don't have much influence on the viewer. And at present, many algorithms in the feature matching technology, which is a fundamental technology in 3D reconstruction, worth improving when the input matching has many outliers in it.
Based on this, firstly, this paper will introduce two improvements to the LMedS algorithm, which is a random sampling based algorithm used in pattern recognition and feature matching. The two new algorithms will improve the LMedS algorithm when the percentage of outliers is high. By constructing data and feed these data to the algorithms, we show that the new two algorithm swill improve the old methods. And then, we show a platform which combines the e-map and the omni-directional video. This includes an IE based omni-directional video player, which is developed by Flash. The platform combines the e-map and the omni-directional in such a way that the video stream from which the play starts to play is given by the e-map and the e-map also shows the current position of the video stream.
引文
[1]高文,陈熙霖.计算机视觉:算法与系统原理[M].北京:清华大学出版社,1998.1-19
[2]Dreschler,Nagel.On the selection of critical points and local curvature extreme of region boundaries for interframe matching[C].International Conference on Pattern Recognition,1982,542-544
[3]S.M Smith,J.M Brady.SUSAN-a new approach to low level image processing[J].International Journal of Computer Vision,1997,23(1):45-78
[4]Kitchen,A.Rosenfeld.Gray-Level Corner Detection[J].Pattern Recognition Letters,1982,1:95-102
[5]A.Noble.Finding corners[J].Image and Vision Computing,1988,6(2):121-128
[6]C.Harris,M.Stephens.A Combined Corner and Edge Detector[C].Proceedings of the Alvey Vision Conference,1988,189-192
[7]R.Tsai,R.Lenz.A new technique for fully autonomous and efficient 3D robotics hand/eye calibration[J].IEEE Trans.Robotics and Automation,1989,5(3):345-358
[8]Z.Zhang.Flexible Camera Calibration by Viewing a Plane from Unknown Orientations[C].Proceedings of the 7~(th)International Conference on Computer Vision,1999,666-673
[9]O.Faugeras,Q.T.Luong,S.Maybank.Camera Self-calibration:Theory and Experiments[C].Proceedings of the 7~(th)European Conference on Computer Vision,1992,588:321-334
[10]S.J.Maybank,O.Faugeras.A Theroy of self-calibration of a Moving Camera[J].Internation Journal of Computer Vision,1992,8(2):123-151
[11]R.Hartley.Euclidean reconstruction from uncalibrated views.Applications of Invariance in Computer Vision[R].Lecture notes in Computer Science,1994,825:237-256
[12]R.Hartley.Euclidean reconstruction and invariants from multiple images[C].Proceedings of the International Conference on Pattern Recognition,1996,339-343
[13] B.Triggs. Auto-calibration and the absolute quadric[C]. Proceedings of the Conference on Computer Vision and Pattern Recognition, 1997,609-614
[14] M.Pollefeys, L.V.Gool, A.Oostertlinck. The modulus constraint: a new constraint for self-calibration[C]. Proceeding of International Conference of Pattern Recognition, 1996, 349-353
[15] C.Tomasi, T.Kanade. Detection and tracking of point features[R]. Technical Report, CMU-CS-91132, Carnegie-Mellon University, 1991
[16] Hartley R. In Defence of the 8-Point Algorithm[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997,19(2):133-137
[17] B.K.P.Horn, B.G.Schunck. Determining optical fiow[J]. Artificial Intelligence, 1981, 17:185-204
[18] Jianbo Shi, Carlo Tomasi. Good features to track[C]. Proc. IEEE Comput. Soc. Conf. Computer Vision and Pattern Recognition, 1994, 593-600
[19] B.Georgescu, P.Meer. Point matching under large deformations and illumination changes[J]. IEEE Trans. Pattern Anal. Machine Intell, 2004,26(6):674-688
[20] Gibson S, Cook J, Hubbold R, Howard T. Accurate camera calibration for off-line, video-based augmented reality[C]. ACM and IEEE International Symposium on Mixed and Augmented Reality, 2002
[21] D.Lowe. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004,60(2):91-110
[22] M.Fischler, R.Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography [J]. Communications of the ACM, 1981,24:381-385
[23] R.Hartley, A.Zisserman. Multiple View Geometry in Computer Vision[M]. Cambridge University Press, London, 2000
[24] O.Faugeras, Q.T.Luong. The Geometry of Multiple Images[M]. MIT Press, Cambridge, Massachusetts, 2001
[25] Jing-Fu Huang, Shang-Hong Lai, Chia-Ming Cheng. Robust Fundamental Matrix Estimation with Accurate Outlier Detection[J]. Journal of information science and engineering, 2007,23:1213-1225
[26] R.Hartley, R.Gupta, T.Chang. Stereo from uncalibrated cameras[C]. Proc. IEEE Conf, on Computer Vision and Pattern Recognition, 1992, 761-764
[27] R.Mohr, F.Veillon, L.Quan. Relative 3D reconstruction using multiple uncalibrated images[C]. Proc. IEEE Conf, on Computer Vision and Pattern Recognition, 1993, 543-548
[28] O.Faugeras. What can be seen in three dimensions with an uncalibrated stereo rig? [C]. Proceedings of the 2~(nd) European Conference on Computer Vision, 1992,563-578
[29] B.Triggs. Matching constraints and the joint image[C]. Proceedings of the 5~(th) International Conference on Computer Vision, 1995, 338-343
[30] Peter Sturm, Bill Triggs. A factorization based algorithm for multi-image projective structure and motion[C]. 4~(th) European Conference on Computer Vision, 1996, 709-720
[31] C.Tomasi, T.Kanade. Shape and motion from image streams under orthography: A factorization method[J]. International Journal of Computer Vision, 1992, 9(2):137-154
[32] C.J.Poelman, T.Kanade. A paraperspective factorization method for shape and motion recovery[C]. Proceedings of the 3~(rd) European Conference on Computer Vision, 1994,97-108
[33] R.Berthilsson, A.Heyden, G.Sparr. Recursive Structure and Motion from Image Sequences Using Shape and Depth Spaces[C]. Proc. Int'l Conf. Computer Vision and Pattern Recognition, 1997,444- 449
[34] A.Heyden, R.Berthilsson, G.Sparr. An Iterative Factorization Method for Projective Structure and Motion from Image Sequences[J]. Image and Vision Computing, 1999,17:981-991
[35] John Oliensis, Richard Hartley. Iterative Extensions of the Sturm/Triggs Algorithm: Convergence and Nonconvergence[J]. IEEE Transactions on pattern Analysis and Machine Intelligencel, 2007,29(12):2217-2233
[36] Han M, Takeo Kanade.T. Creating 3D models with uncalibrated cameras[C]. Proceedings of IEEE Computer Society Workshop on the Application of Computer Vision, 2000,178-185
[37] A.Heyden, K.Astrom. Euclidean reconstruction from image sequences with varying and unknown focal length and principle point[C]. EBEE Conf, on Computer Vision and Pattern Recognition, 1997,438-443
[38] M.Pollefeys, R.Koch, L.VanGool. Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters[C]. International Conference on Computer Vision, 1998,90-95
[39] Mei Han, Takeo Kanade. A perspective factorization method for Euclidean reconstruction with uncalibrated camera[J]. The Journal of Visualization and Computer Animation, 2002,13: 211-223
[40] L.de Agapito, R.I.Hartley and E.Hayman. Linear self-calibration of a rotating and zooming camera[C]. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1999,1015-1021
[2]Dreschler,Nagel.On the selection of critical points and local curvature extreme of region boundaries for interframe matching[C].International Conference on Pattern Recognition,1982,542-544
[3]S.M Smith,J.M Brady.SUSAN-a new approach to low level image processing[J].International Journal of Computer Vision,1997,23(1):45-78
[4]Kitchen,A.Rosenfeld.Gray-Level Corner Detection[J].Pattern Recognition Letters,1982,1:95-102
[5]A.Noble.Finding corners[J].Image and Vision Computing,1988,6(2):121-128
[6]C.Harris,M.Stephens.A Combined Corner and Edge Detector[C].Proceedings of the Alvey Vision Conference,1988,189-192
[7]R.Tsai,R.Lenz.A new technique for fully autonomous and efficient 3D robotics hand/eye calibration[J].IEEE Trans.Robotics and Automation,1989,5(3):345-358
[8]Z.Zhang.Flexible Camera Calibration by Viewing a Plane from Unknown Orientations[C].Proceedings of the 7~(th)International Conference on Computer Vision,1999,666-673
[9]O.Faugeras,Q.T.Luong,S.Maybank.Camera Self-calibration:Theory and Experiments[C].Proceedings of the 7~(th)European Conference on Computer Vision,1992,588:321-334
[10]S.J.Maybank,O.Faugeras.A Theroy of self-calibration of a Moving Camera[J].Internation Journal of Computer Vision,1992,8(2):123-151
[11]R.Hartley.Euclidean reconstruction from uncalibrated views.Applications of Invariance in Computer Vision[R].Lecture notes in Computer Science,1994,825:237-256
[12]R.Hartley.Euclidean reconstruction and invariants from multiple images[C].Proceedings of the International Conference on Pattern Recognition,1996,339-343
[13] B.Triggs. Auto-calibration and the absolute quadric[C]. Proceedings of the Conference on Computer Vision and Pattern Recognition, 1997,609-614
[14] M.Pollefeys, L.V.Gool, A.Oostertlinck. The modulus constraint: a new constraint for self-calibration[C]. Proceeding of International Conference of Pattern Recognition, 1996, 349-353
[15] C.Tomasi, T.Kanade. Detection and tracking of point features[R]. Technical Report, CMU-CS-91132, Carnegie-Mellon University, 1991
[16] Hartley R. In Defence of the 8-Point Algorithm[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1997,19(2):133-137
[17] B.K.P.Horn, B.G.Schunck. Determining optical fiow[J]. Artificial Intelligence, 1981, 17:185-204
[18] Jianbo Shi, Carlo Tomasi. Good features to track[C]. Proc. IEEE Comput. Soc. Conf. Computer Vision and Pattern Recognition, 1994, 593-600
[19] B.Georgescu, P.Meer. Point matching under large deformations and illumination changes[J]. IEEE Trans. Pattern Anal. Machine Intell, 2004,26(6):674-688
[20] Gibson S, Cook J, Hubbold R, Howard T. Accurate camera calibration for off-line, video-based augmented reality[C]. ACM and IEEE International Symposium on Mixed and Augmented Reality, 2002
[21] D.Lowe. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004,60(2):91-110
[22] M.Fischler, R.Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography [J]. Communications of the ACM, 1981,24:381-385
[23] R.Hartley, A.Zisserman. Multiple View Geometry in Computer Vision[M]. Cambridge University Press, London, 2000
[24] O.Faugeras, Q.T.Luong. The Geometry of Multiple Images[M]. MIT Press, Cambridge, Massachusetts, 2001
[25] Jing-Fu Huang, Shang-Hong Lai, Chia-Ming Cheng. Robust Fundamental Matrix Estimation with Accurate Outlier Detection[J]. Journal of information science and engineering, 2007,23:1213-1225
[26] R.Hartley, R.Gupta, T.Chang. Stereo from uncalibrated cameras[C]. Proc. IEEE Conf, on Computer Vision and Pattern Recognition, 1992, 761-764
[27] R.Mohr, F.Veillon, L.Quan. Relative 3D reconstruction using multiple uncalibrated images[C]. Proc. IEEE Conf, on Computer Vision and Pattern Recognition, 1993, 543-548
[28] O.Faugeras. What can be seen in three dimensions with an uncalibrated stereo rig? [C]. Proceedings of the 2~(nd) European Conference on Computer Vision, 1992,563-578
[29] B.Triggs. Matching constraints and the joint image[C]. Proceedings of the 5~(th) International Conference on Computer Vision, 1995, 338-343
[30] Peter Sturm, Bill Triggs. A factorization based algorithm for multi-image projective structure and motion[C]. 4~(th) European Conference on Computer Vision, 1996, 709-720
[31] C.Tomasi, T.Kanade. Shape and motion from image streams under orthography: A factorization method[J]. International Journal of Computer Vision, 1992, 9(2):137-154
[32] C.J.Poelman, T.Kanade. A paraperspective factorization method for shape and motion recovery[C]. Proceedings of the 3~(rd) European Conference on Computer Vision, 1994,97-108
[33] R.Berthilsson, A.Heyden, G.Sparr. Recursive Structure and Motion from Image Sequences Using Shape and Depth Spaces[C]. Proc. Int'l Conf. Computer Vision and Pattern Recognition, 1997,444- 449
[34] A.Heyden, R.Berthilsson, G.Sparr. An Iterative Factorization Method for Projective Structure and Motion from Image Sequences[J]. Image and Vision Computing, 1999,17:981-991
[35] John Oliensis, Richard Hartley. Iterative Extensions of the Sturm/Triggs Algorithm: Convergence and Nonconvergence[J]. IEEE Transactions on pattern Analysis and Machine Intelligencel, 2007,29(12):2217-2233
[36] Han M, Takeo Kanade.T. Creating 3D models with uncalibrated cameras[C]. Proceedings of IEEE Computer Society Workshop on the Application of Computer Vision, 2000,178-185
[37] A.Heyden, K.Astrom. Euclidean reconstruction from image sequences with varying and unknown focal length and principle point[C]. EBEE Conf, on Computer Vision and Pattern Recognition, 1997,438-443
[38] M.Pollefeys, R.Koch, L.VanGool. Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters[C]. International Conference on Computer Vision, 1998,90-95
[39] Mei Han, Takeo Kanade. A perspective factorization method for Euclidean reconstruction with uncalibrated camera[J]. The Journal of Visualization and Computer Animation, 2002,13: 211-223
[40] L.de Agapito, R.I.Hartley and E.Hayman. Linear self-calibration of a rotating and zooming camera[C]. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1999,1015-1021