视频序列中基于边缘信息的运动分割方法
|
摘要
近年来，随着MPEG-4压缩标准的制定和越来越多数字视频形式的出现，人们对自动分析视频技术的需求也越来越迫切。在MPEG-4压缩标准中，为了实现视频内容的交互功能，视频序列的每一帧都由视频对象面(VOP)决定，而为了生成这些视频对象面，需要对视频序列中的运动对象进行分割。分析视频的第一步就是把视频序列分割成代表真实世界中不同物体的区域，而且每个区域都遵循着各自对应物体的运动方式。实际上，真正有效的分割技术应该能够提供明确的物体间的划分，以及它们之间相对的深度信息。
运动分割是许多视频分析与应用的必要过程，其目的是把视频序列中的各帧图像按照不同的运动分割成不同的区域，从而分离出图像中的运动对象。本文运用了一种基于边缘信息的运动分割方法，来处理包含前后景两个运动的视频序列。首先用Canny算法提取较为连续的物体边缘，由边缘的法线方向追踪出法向运动，然后对两个运动分别作最大似然法，交替估计出运动和各边缘所属运动的概率，反复迭代直至收敛；再以这些边缘为严格界限，用区域增长法将整幅图像分割成若干区域，最后由于区域标记过程交替估计出正确的运动层次，以确定最后的分割。实验结果表明这种改进的运动跟踪和区域标记方法能够得到很好的分割效果。
Following the standard of MPEG-4 and the increasing availability of video in digital form, there is growing demand for methods of automatically analyzing video. In the object-based standard of MPEG-4, in order to support content-based functions, each frame of video should be represented in terms of video object planes (VOP).The first stage is the segmentation of the frames into regions representing different real world objects which have different motion. Exactly, the ideal segmentation should provide a clear partition of objects and the relative depth ordering of each object.
Motion segmentation is the necessary stage of most video analysis implements, and it divides frames into different regions according different motion so that those motion objects can be abstracted. This paper presents an improved theoretical framework for motion segmentation based on the motion of tracked edges. Firstly, abstract these edges of objects by Canny algorithm, then tracking edges along normal direction to get normal motion. Secondly, applying Expectation-Maximization algorithm to the two motions alternately to calculate the motions and the responsibility of every edges, repeat the step until it is convergent. Finally segment the sequence into regions by region growing algorithm, and get the final result by region labeling. The practical result has proved the validity of the improved motion tracking and region labeling method.
运动分割是许多视频分析与应用的必要过程，其目的是把视频序列中的各帧图像按照不同的运动分割成不同的区域，从而分离出图像中的运动对象。本文运用了一种基于边缘信息的运动分割方法，来处理包含前后景两个运动的视频序列。首先用Canny算法提取较为连续的物体边缘，由边缘的法线方向追踪出法向运动，然后对两个运动分别作最大似然法，交替估计出运动和各边缘所属运动的概率，反复迭代直至收敛；再以这些边缘为严格界限，用区域增长法将整幅图像分割成若干区域，最后由于区域标记过程交替估计出正确的运动层次，以确定最后的分割。实验结果表明这种改进的运动跟踪和区域标记方法能够得到很好的分割效果。
Following the standard of MPEG-4 and the increasing availability of video in digital form, there is growing demand for methods of automatically analyzing video. In the object-based standard of MPEG-4, in order to support content-based functions, each frame of video should be represented in terms of video object planes (VOP).The first stage is the segmentation of the frames into regions representing different real world objects which have different motion. Exactly, the ideal segmentation should provide a clear partition of objects and the relative depth ordering of each object.
Motion segmentation is the necessary stage of most video analysis implements, and it divides frames into different regions according different motion so that those motion objects can be abstracted. This paper presents an improved theoretical framework for motion segmentation based on the motion of tracked edges. Firstly, abstract these edges of objects by Canny algorithm, then tracking edges along normal direction to get normal motion. Secondly, applying Expectation-Maximization algorithm to the two motions alternately to calculate the motions and the responsibility of every edges, repeat the step until it is convergent. Finally segment the sequence into regions by region growing algorithm, and get the final result by region labeling. The practical result has proved the validity of the improved motion tracking and region labeling method.
引文
[1]J. M. Odobez and P. Bouthemy. Separation of moving regions from background in an image sequence acquired with a mobile camera. In Video Data Compression for Multimedia ComputingKluwer academic Publisher, 1997, pages 283-311.
[2]Y.Weiss and E. H. Adelson. A unified mixture framework for motion segmentation: Incorporating spatial coherence and estimating the number of models. In Proe. IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, June 1996, P321-326.
[3]徐静波。MPEG—4标准与多媒体通讯。信息工程学院学报，第8卷第4期 1999
[4]T. Sikora The MPEG-4 Video Standard Verification Model IEEE Trans, CSVT, Feb 1997 Paul Alexander Smith Vol. 7, No. 1.
[5]Laszlo Czuni and Tamas Sziranyi. Motion Segmentation and Tracking with Edge Relaxation and Optimization using Fully Parallel Methods in the Cellular Nonlinear Network Architecture Real-Time Imaging 7, 2001. P77-95
[6]S. Hsu, P. Anandan, and S. Peleg. Accurate computation of optical flow by using layered motion representations. In International Conference on Pattern Recognition, October 1994.
[7]S. Ayer and H. S. Sawhney. Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding. International Conference on Computer Vision, Cambridge, June 1995.
[8]D. Sinclair. Voronoi seeded colour image segmentation. AT&T Laboratories Cambridge, submitted for publication, 1999.
[9]Mech R, Wollborn M. Automatic segmentationof moving objeets. MPEG98/m3187,1998
[10]Y. Weiss and E. H. Adelson. A unified mixture framework for motion segmentation: Incorporating spatial coherence and estimating the number of models. IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, June 1996.
[11]Edge-based Motion Segmentation Jesus College August 2001
[12]Canny J. A computational approach to edge detection. IEEE Trans on Patten Analysis and Machine Intelligence, 1986
[13]F. Dufaux, F. Moseheni, and A. Lippman. Spatio-temporal segmentation based on motion and static segmentation. In Proc. International Conference on Image Processing, Washington DC, USA, October 1995.
[14]T. Drummond and R. Cipolla. Application of Lie algebras to visual servoing. International Journal of Computer Vision, June 2000.
[15]F. Moseheni and S Bhattacharjee. Robust region merging for spatio-temporal segmentation. In Proc. International Conference on Image Processing Lausanne,
Switzerland, September 1996.
[16]F. Moscheni and F. Dufaux. Region merging based on robust statistical testing. In Proc. SPIE Visual Communications and Image Processing, volume 2727, Orlando, Florida, USA, March 1996.
[17]B. K. P. Horn and B. G. Schunk. Determining optical flow. Artificial Intelligence, August 1981, 17(1-3):185-203.
[18]贾志科，崔慧娟，唐昆，冯重熙，用于运动位移估值的新匹配准则。清华大学学报(自然科学版)1999年第1期 No．1
[19]F. Moscheni and F. Dufaux. Region merging based on robust statistical testing. In SPIE Proc. Visual Communications and Image Processing, Orlando, Florida,USA, March 1996.
[20]N. Brady and N. O' Connor. Object detection and tracking using an EM-based motion estimation and segmentation framework. IEEE International Conferenee on Image Processing, volume 1, Lausanne, Switzerland, September 1996, pages 925-928.
[21]N. Ueda and R. Nakano. Deterministic annealing EM algorithm. Neural Networks, April 1998, pages 271-282.
[22]严志达，许以超。Lie群以及Lie代数。高等教育出版社1985，P11—38．
[23]V. S. Varadarajan. Lie Groups, Lie Algebras, and their Representations, volume 102 of Graduate Texts in Mathematics. Springer-Verlag, New York, 1984.
[24]M. J. Black and P. Anandan. Robust dynamic motion estimation over time. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, USA, June 1991, pages 296-302.
[25]史立，张兆扬，马然。 基于运动跟踪匹配计术的视频对象提取。通信学报， Nov 2001 V01．22 No．11
[26]ISO/IEC JTC1/SC29/WGll, information Technology-Coding of Audio-Visual Objects Part 2 Visual/Amendment Dee. 1999, P452-462
[27]杨勇，王桥，吴乐南。基于标号场的光流法二维运动估计。电子科学学刊，2001年，第23卷 第12期
[28]季白杨，陈纯，钱英。视频分割技术的发展。计算机研究与发 展第38卷第1期2001
[29]R. Koenen, Overview of the MPEG-4 Standard, ISO/IEC N2725, Mar. 1999.
[30]M. J. Black and P. Anandau. A model for the detection of motion over time. In Proc. 3rd International Conference on Computer Vision, Japan, December 1990, pages 33-37.
[31]黄波，杨勇，王桥，吴乐南．一种基于时空联合的视频分割算法．电子学报，Nov 2001 Vol．29 No．11
[32]M. Gelgon and P. Bouthemy. Determining a structured spatial-temporal representation of video content for visualization and indexing. In Computer 98
volume 1407 of Lecture Notes in Computer Science, June 1998, pages 595-609.
[33]C. Gu, M.C. Lee Semiautomatic segmentation and tracking of semantic video objects. IEEE Trans. Oct. 1998 pages 572-584
[34]T. Meier and K. N. Ngan. Automatic segmentation of moving objects for video object plane generation. IEEE Transactions on Circuits and Systems for Video Technology, September 1998, P525-538.
[35]Jane D S, Choi H I. Active models for tracking moving objects. Pattern Recognition, 2000,33(7) pages 1135-1146
[36]R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision.Cambridge University Press, 2000.
[37]P. J. Huber. Robust Statistics. Wiley Series in Probability and Mathematical Statistics.John Wiley & Sons, New York, 1981.
[38]田村秀行等著，赫荣威等编译，计算机图像处理技术．北师大出版社 1995．
[39]李月景编著，图像识别技术及其应用。北京机械工业出版社 1985
[40]P. Smith, T. Drummond, and R. Cipolla. Edge tracking for motion segmentation and depth ordering. In Proc. 10th British Machine Vision Conference, Nottingham, UK, September 1999, pages 584-593.
[41]夏德深 傅德胜，现代图像处理技术与应用。南京东南大学出版社 1997
[42]何斌，马天予，王运坚，朱红莲。Visual C++数字图像处理。北京人民邮电出版社，2001．4
[43]张文涛，李晓峰，李在铭。可视动目标逐步求精分割技术研究。系统工程与电子计术，第22卷第6期，2000
[44]C. G. Harris and M. Stephens. A combined corner and edge detector. In Proc. 4th Alvey Vision Conference, Manchester, UK, August 1988, pages 147-151.
[45]D. Marr and E. C. Hildreth. Theory of edge detection. In Proc. Royal Society of London, 1980, volume 207, pages 187-217.
[46]L. Paletta, D. Sinclair, and A. Pinz. Classification of edges under motion. In Proc. 10th Scandinavian Conference on Image Analysis, Lappeenranta, Finland, June 1997, pages 613-620.
[47]卡斯尔曼著，朱志刚 林学訚 石定机译。数字图像处理北京电子工业出版社 2002
[48]拉斐尔·冈萨雷斯著，数字图像处理(第二版)(国外电子与通信教材系列)北京电子工业出版社 2003
[49]沈庭芝，方子文，数字图像处理及模式识别，北京理工大学出版社 1998
[50]P. Bouthemy. A maximum likelihood framework for determining moving edges. IEEE Transactions on Pattern Analysis and Machine Intelligence, May 1989, pages 499-511.
[51]徐建华编著，图像处理与分析。北京科学出版社 1992
[52]J. Y. A. Wang and E. H. Adelson. Representing moving images with layers. IEEE Transactions on Image Processing, September 1994, P625-638.
[53]D. Tweed and A. Calway. Integrated segmentation and depth ordering of motion layers in image sequences. In Proc. 11th British Machine Vision Conference, Bristol, UK, September 2000, pages 322-331.
[54]刘新春，周建宇，邹谋炎。一种面向对象的视频分割算法。电子信息学报 Dec 2001．Vol.23 No．12
[55]张旭东，王德生，彭应宁．高效多分层半像素精度块匹配运动估计算法。清华大学学报(自然科学版)1999年 第39卷 第5期 VOL．39 No．5
[56]A. P. Dempster, H. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of Royal Statistical Society Series B, 1977. pages 1-38
[2]Y.Weiss and E. H. Adelson. A unified mixture framework for motion segmentation: Incorporating spatial coherence and estimating the number of models. In Proe. IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, June 1996, P321-326.
[3]徐静波。MPEG—4标准与多媒体通讯。信息工程学院学报，第8卷第4期 1999
[4]T. Sikora The MPEG-4 Video Standard Verification Model IEEE Trans, CSVT, Feb 1997 Paul Alexander Smith Vol. 7, No. 1.
[5]Laszlo Czuni and Tamas Sziranyi. Motion Segmentation and Tracking with Edge Relaxation and Optimization using Fully Parallel Methods in the Cellular Nonlinear Network Architecture Real-Time Imaging 7, 2001. P77-95
[6]S. Hsu, P. Anandan, and S. Peleg. Accurate computation of optical flow by using layered motion representations. In International Conference on Pattern Recognition, October 1994.
[7]S. Ayer and H. S. Sawhney. Layered representation of motion video using robust maximum-likelihood estimation of mixture models and MDL encoding. International Conference on Computer Vision, Cambridge, June 1995.
[8]D. Sinclair. Voronoi seeded colour image segmentation. AT&T Laboratories Cambridge, submitted for publication, 1999.
[9]Mech R, Wollborn M. Automatic segmentationof moving objeets. MPEG98/m3187,1998
[10]Y. Weiss and E. H. Adelson. A unified mixture framework for motion segmentation: Incorporating spatial coherence and estimating the number of models. IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, June 1996.
[11]Edge-based Motion Segmentation Jesus College August 2001
[12]Canny J. A computational approach to edge detection. IEEE Trans on Patten Analysis and Machine Intelligence, 1986
[13]F. Dufaux, F. Moseheni, and A. Lippman. Spatio-temporal segmentation based on motion and static segmentation. In Proc. International Conference on Image Processing, Washington DC, USA, October 1995.
[14]T. Drummond and R. Cipolla. Application of Lie algebras to visual servoing. International Journal of Computer Vision, June 2000.
[15]F. Moseheni and S Bhattacharjee. Robust region merging for spatio-temporal segmentation. In Proc. International Conference on Image Processing Lausanne,
Switzerland, September 1996.
[16]F. Moscheni and F. Dufaux. Region merging based on robust statistical testing. In Proc. SPIE Visual Communications and Image Processing, volume 2727, Orlando, Florida, USA, March 1996.
[17]B. K. P. Horn and B. G. Schunk. Determining optical flow. Artificial Intelligence, August 1981, 17(1-3):185-203.
[18]贾志科，崔慧娟，唐昆，冯重熙，用于运动位移估值的新匹配准则。清华大学学报(自然科学版)1999年第1期 No．1
[19]F. Moscheni and F. Dufaux. Region merging based on robust statistical testing. In SPIE Proc. Visual Communications and Image Processing, Orlando, Florida,USA, March 1996.
[20]N. Brady and N. O' Connor. Object detection and tracking using an EM-based motion estimation and segmentation framework. IEEE International Conferenee on Image Processing, volume 1, Lausanne, Switzerland, September 1996, pages 925-928.
[21]N. Ueda and R. Nakano. Deterministic annealing EM algorithm. Neural Networks, April 1998, pages 271-282.
[22]严志达，许以超。Lie群以及Lie代数。高等教育出版社1985，P11—38．
[23]V. S. Varadarajan. Lie Groups, Lie Algebras, and their Representations, volume 102 of Graduate Texts in Mathematics. Springer-Verlag, New York, 1984.
[24]M. J. Black and P. Anandan. Robust dynamic motion estimation over time. In Proc. IEEE Conference on Computer Vision and Pattern Recognition, USA, June 1991, pages 296-302.
[25]史立，张兆扬，马然。 基于运动跟踪匹配计术的视频对象提取。通信学报， Nov 2001 V01．22 No．11
[26]ISO/IEC JTC1/SC29/WGll, information Technology-Coding of Audio-Visual Objects Part 2 Visual/Amendment Dee. 1999, P452-462
[27]杨勇，王桥，吴乐南。基于标号场的光流法二维运动估计。电子科学学刊，2001年，第23卷 第12期
[28]季白杨，陈纯，钱英。视频分割技术的发展。计算机研究与发 展第38卷第1期2001
[29]R. Koenen, Overview of the MPEG-4 Standard, ISO/IEC N2725, Mar. 1999.
[30]M. J. Black and P. Anandau. A model for the detection of motion over time. In Proc. 3rd International Conference on Computer Vision, Japan, December 1990, pages 33-37.
[31]黄波，杨勇，王桥，吴乐南．一种基于时空联合的视频分割算法．电子学报，Nov 2001 Vol．29 No．11
[32]M. Gelgon and P. Bouthemy. Determining a structured spatial-temporal representation of video content for visualization and indexing. In Computer 98
volume 1407 of Lecture Notes in Computer Science, June 1998, pages 595-609.
[33]C. Gu, M.C. Lee Semiautomatic segmentation and tracking of semantic video objects. IEEE Trans. Oct. 1998 pages 572-584
[34]T. Meier and K. N. Ngan. Automatic segmentation of moving objects for video object plane generation. IEEE Transactions on Circuits and Systems for Video Technology, September 1998, P525-538.
[35]Jane D S, Choi H I. Active models for tracking moving objects. Pattern Recognition, 2000,33(7) pages 1135-1146
[36]R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision.Cambridge University Press, 2000.
[37]P. J. Huber. Robust Statistics. Wiley Series in Probability and Mathematical Statistics.John Wiley & Sons, New York, 1981.
[38]田村秀行等著，赫荣威等编译，计算机图像处理技术．北师大出版社 1995．
[39]李月景编著，图像识别技术及其应用。北京机械工业出版社 1985
[40]P. Smith, T. Drummond, and R. Cipolla. Edge tracking for motion segmentation and depth ordering. In Proc. 10th British Machine Vision Conference, Nottingham, UK, September 1999, pages 584-593.
[41]夏德深 傅德胜，现代图像处理技术与应用。南京东南大学出版社 1997
[42]何斌，马天予，王运坚，朱红莲。Visual C++数字图像处理。北京人民邮电出版社，2001．4
[43]张文涛，李晓峰，李在铭。可视动目标逐步求精分割技术研究。系统工程与电子计术，第22卷第6期，2000
[44]C. G. Harris and M. Stephens. A combined corner and edge detector. In Proc. 4th Alvey Vision Conference, Manchester, UK, August 1988, pages 147-151.
[45]D. Marr and E. C. Hildreth. Theory of edge detection. In Proc. Royal Society of London, 1980, volume 207, pages 187-217.
[46]L. Paletta, D. Sinclair, and A. Pinz. Classification of edges under motion. In Proc. 10th Scandinavian Conference on Image Analysis, Lappeenranta, Finland, June 1997, pages 613-620.
[47]卡斯尔曼著，朱志刚 林学訚 石定机译。数字图像处理北京电子工业出版社 2002
[48]拉斐尔·冈萨雷斯著，数字图像处理(第二版)(国外电子与通信教材系列)北京电子工业出版社 2003
[49]沈庭芝，方子文，数字图像处理及模式识别，北京理工大学出版社 1998
[50]P. Bouthemy. A maximum likelihood framework for determining moving edges. IEEE Transactions on Pattern Analysis and Machine Intelligence, May 1989, pages 499-511.
[51]徐建华编著，图像处理与分析。北京科学出版社 1992
[52]J. Y. A. Wang and E. H. Adelson. Representing moving images with layers. IEEE Transactions on Image Processing, September 1994, P625-638.
[53]D. Tweed and A. Calway. Integrated segmentation and depth ordering of motion layers in image sequences. In Proc. 11th British Machine Vision Conference, Bristol, UK, September 2000, pages 322-331.
[54]刘新春，周建宇，邹谋炎。一种面向对象的视频分割算法。电子信息学报 Dec 2001．Vol.23 No．12
[55]张旭东，王德生，彭应宁．高效多分层半像素精度块匹配运动估计算法。清华大学学报(自然科学版)1999年 第39卷 第5期 VOL．39 No．5
[56]A. P. Dempster, H. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of Royal Statistical Society Series B, 1977. pages 1-38
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |