基于水平集方法的运动对象分割
|
摘要
光流场反映了图像上每一点灰度的变化趋势。它计算的可靠性成了计算机视觉领域面临的很大挑战,而变分方法是目前为止用来实现光流计算的最成功的方法之一。本文的研究内容之一是基于不同的光照条件和运动类型假设条件,详细论述了Horn-Schunck光流计算模型以及基于图像亮度梯度常值、图像亮度Hessian矩阵常值、图像亮度梯度的模常值、图像亮度拉普拉斯算子常值和图像亮度Hessian矩阵的行列式常值前提假设的六种光流计算模型,并对这六种模型进行了实验验证和结果比较。运动对象分割作为图像分割的重要分支,在视频跟踪,视频会议、可视电话、现场监控等方面得到了广泛的应用。本文的另一研究内容是在前面做的光流计算研究的基础上,采用光流法与水平集方法相结合的思想,将Horn-Schunck光流计算模型、图像亮度梯度常值模型以及图像亮度Hessian矩阵常值模型分别与基于区域信息的分割模型中的Chan-Vese模型相结合,提出了三种新的运动对象分割模型,并详细说明了分割原理及其数值算法,最后进行了实验论证。新的运动对象分割模型的提出,克服了减背景法在背景与运动物体部分灰度值接近时,导致运动目标产生缺口、空洞和分离的缺点。同时,新的运动对象分割模型,可以在图像任意位置定义初始轮廓线,实现了运动对象的自动分割。
Optic flow describes the displacement field in an image sequence. Its reliable computation constitutes one of the main challenges in computer vision, and variational methods belong to the most successful techniques for achieving this goal. One of the research content of this article is optic computation based on the assumption of different light conditions and different motion types. Six optic computation models, that is Horn-Schunck optic computation model, models based on constancy assumptions of the gradient, the Hessian, the gradient magnitude, the Laplacian and the Hessian determinant, are detailed explained and illustrated. Integrated comparison of the lab results for these six models are performed. Motion segmentation, as an important branch of image segmentation, is widely used in video tracking, video meeting, visual phone and live monitoring. Based on the study of optic computation models, through combining the level set method with the optic computation models, another research content of this article is combining the Chan-Vese model with three optic computation models, which are constancy assumptions on the brightness, the gradient and the Hessian. Three new motion segmentation models are initiated, and their segmentation theories and numeric arithmetic are fully discussed. Also their impacts are illustrated by experiments. Background subtracting method for motion segmentation sometimes has rupture, blank areas and partition when the color of the background and part of the moving object are nearly the same. New motion segmentation models avoid these disadvantages. Meanwhile, initial active contours can be displayed in any position in image, and the automated segmentation for moving object is implemented.
Optic flow describes the displacement field in an image sequence. Its reliable computation constitutes one of the main challenges in computer vision, and variational methods belong to the most successful techniques for achieving this goal. One of the research content of this article is optic computation based on the assumption of different light conditions and different motion types. Six optic computation models, that is Horn-Schunck optic computation model, models based on constancy assumptions of the gradient, the Hessian, the gradient magnitude, the Laplacian and the Hessian determinant, are detailed explained and illustrated. Integrated comparison of the lab results for these six models are performed. Motion segmentation, as an important branch of image segmentation, is widely used in video tracking, video meeting, visual phone and live monitoring. Based on the study of optic computation models, through combining the level set method with the optic computation models, another research content of this article is combining the Chan-Vese model with three optic computation models, which are constancy assumptions on the brightness, the gradient and the Hessian. Three new motion segmentation models are initiated, and their segmentation theories and numeric arithmetic are fully discussed. Also their impacts are illustrated by experiments. Background subtracting method for motion segmentation sometimes has rupture, blank areas and partition when the color of the background and part of the moving object are nearly the same. New motion segmentation models avoid these disadvantages. Meanwhile, initial active contours can be displayed in any position in image, and the automated segmentation for moving object is implemented.
引文
1Pal N R,Pal S K.A Review on Image Segmentation Techniques[J].Pattern Recognition,1993,26(9):1277-1294
2Meier T,Ngan K N.Automatic Segmentation of Moving Objects for Video Object Plane Generation[J].IEEE Trans.Circuits System for Video Technology,1998,8(5):525-538
3邓玉春,姜昱明,张建荣.视频序列图像中运动对象分割综述.计算机应用研究,2005,1:8-11
4B KP Horn,B G Schunck.Determining Optical Flow[J].Artificial Intelligence,1981,17:185-203
5B O Lucas,Z Kanade.An IterativeImage Registration Technique with an Application to Stereo Vision[C].Processing of the International Joint Conference on Artificial Intelligence,1981
6H-H Nagel.Displacement Vectors Derived from Second-Order Intensity Variations in Image Sequence[J].Computer Graphics and Image Processing,1983,21:299-324
7M.J.Black and P.Anandan.Robust dynamic motion estimation over time[J].In Proc.1991 IEEE Computer SocietyConference on Computer Vision and PatternRecognition,Maul,HI,1991,292-302
8E.Memin and P.Perez.Dense estimation and object-based segmentation of the optical flow with robust techniques[J].IEEE Transactions on Image Processing,1998,7(5):703-719
9C.Schnorr.On functionals with greyvalue-controlled smoothness terms for determining optical flow[J].IEEE Transactions on PatternAnalysis and kIachine Intelligence,1993,15:1074-1079
10H.-H.Nagel and W.Enkelmann.An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1986,8:565-593
11L.Alvarez,J.Weickert,and J.Sanchez.Reliable estimation of dense optical flow fields with large displacements[J].International JournalofComputerVision,2000,39(1):41-56
12H.-H.Nagel.Constraints for the estimation of displacement vector fields from image sequences[J].In Proc.Eighth International Joint Conference on Artificial Intelligence,volume 2,Karlsruhe,West Germany,1983,945-951
13D.Shulman and J.Herve.Regularization of discontinuous flow fields[J].In Proc.Workshop on Visual Motion,Irvine,CA,1989,81-90
14F.Heitz and P.Bouthemy.Multimodal estimation of discontinuous optical flow using Markov random fields[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 1993,15(12):1217-1232
15 P. Nesi. Variational approach to optical flow estimation managing discontinuities [J]. Image and Vision Computing, 1993, 11(7): 419-439
16 J. Weickert and C. Schnorr. A theoretical framework for convex regularizes in PDE-based computation of image motion [J]. International Journal of Computer Vision, 2001, 45(3):245-264
17 H.-H. Nagel. Extending the ' oriented smoothness constraint' into the temporal domain and the estimation of derivatives of optical flow [J]. In O. Faugeras, editor, Computer Vision - ECCV ' 90, volume 427 of Lecture Notes in Computer Science, Springer, Berlin, 1990, 139-148
18 J. Weickert and C. Schnorr. Variational optic flow computation with a spatio-temporal smoothness constraint [J]. Journal of Mathematical Imaging and Vision, 2001, 14(3):245-255
19 E P Ong. Robust Optical Flow Computation Based on Least Median of Squares Regression [J]. Computer Vision, 1999, 31(1): 51-82
20 A B Hadiashar, D Suter. Robust Optical Flow Computation [J]. Computer Vision, 1998, 29(1): 59-77
21 Roland Mech, Michael Wollborn. A Noise Robust Method for 2D Shape Estimation of Moving Objects in Video Sequences Considering a Moving Camera [J]. Signal Processing, 1998, 66(2):203-217
22 Ramesh Jain. Dynamic Scene Analysis Using Pixel-based Processes [J]. Computer, 1981, 14(8): 12-18
23 S N Jayaramamurthy, Ramesh Jain. An Approach to the Segmentation of Textured Dynamic Scenes [J]. Computer Graphics and Image Processing, 1983, 21: 239-261
24 Robert Thoma, Matthias Bierling. Motion Compensating Interpolation Considering Covered and Uncovered Background [J]. Signal Processing: Image Communication, 1989, 1: 191-212
25 Til Aach, Andre Kaup, et al. Statistical Model-based Change Detection in Moving Video [J]. Signal Processing, 1993, 31(2): 165-180
26 A Div G. Determining Three-Dimensional Motion and Structure form Optical Flow Generated by Several Moving Objects [J]. IEEE Trans. on Pattern Analysis Machine Intelligence, 1985, 7(4): 384-401
27 Wang J, Adelson E. Representing Moving Images with Layers [J]. IEEE Trans. on Image Processing, 1994, 3(5): 625-638
28 S Geman, D Geman. Stochastic Relaxation, Gipps Distributions and the Bayesian Restoration of Images [J]. IEEETrans. on Pattern Analysis and Machine Intelligence, 1984, 6(6): 721-741
29 Murray D W, Buxton B F. Scene Segmentation form Visual Motion Using Global Optimization [J]. IEEE Trans. on Pattern Analysis and Machine Intelligence, 1987, 9(2): 220-228
30 Michael M Chang, A Murat Tekalp, M Ibrahim Sezan. An Algorithm for Simultaneous Motion Estimation and Segmentation [C]. IEEE Internal Conference on Acoustics, Speech and Signal Processing (ICASSP' 94), Adelaide, Australia, 1994. 221-224
31 Patrick Bouthemy, Edouard Francois. Motion Segmentation and Qualitative Dynamic Scene Analysis from an Image Sequence [J]. International Journal of Computing Vision, 1993, 10 (2): 157-182
32 S. Osher and J. A. Sethian. Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations[J]. Journal of Computational Physics, 1988: vol. 79, pp. 12-49
33 V. Caselles, F. Catte, T. Coll, and F. Dibos. A geometric model for active contours[J]. Numerische Mathematik, 1993: vol. 66, pp. 1-31
34 R. Malladi, J. A. Sethian, and B. C. Vemuri. Shape modeling with front propagation: A level set approach[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1995: vol. 17, pp. 158-175
35 S. Kichenassamy, A. Kumar, P. Olver, A. Tannenbaum, and A. Yezzi. Gradient flows and geometric active contour models[R]. in the Proceedings of IEEE International Conference in Computer Vision, pp. 810-815, Cambridge, MA, USA, 1995
36 V. Caselles, R. Kimmel, and G. Sapiro. Geodesic active contours[A]. In the Proceedings of International Conference on Computer Vision[C]. Cambridge, MA, USA, 1995: pp. 694-699
37 Zhao H. K., Chan T., Merriman B. and Osher S.. A variational level set approach to multiphase motion. J. Comput. Phys., 1996, 127: 179-195
38 李俊. 基于曲线演化的图像分割方法及应有研究[D]. 上海：上海交通大学,2002.
39 D. Mumford, J. Shah. Boundary detection by minimizing functional [A]. Proc. IEEE Conf. Computer Vision and Pattern Recognition[C]. San Francisco, CA, 1985:188-211
40 F.T. Chan, L. Vese. Active contours without edges[J]. IEEE Transactions on Image Processing, 2001:10(2), 266-277
41 L. A. Vese, T. F. Chan. A Multiphase Level Set Framework for Image Segmentation Using the Mumford and Shah Model [J] . International Journal of Computer Vision, 2002, 50(3): 271-293
42 Comes J and Fangeras O. Reconciling distance function and level sets[J]. Visual Communic and Image Representation, 2000:vol. 11:209-223
43 D. Chop. Computing minimal surfaces via Level Set curvature flow[J]. Journal of Computational Physic,1993:106:77-91
44J.Sethian.A fast marching level set method for monotonically advancing fronts[J].In Proc.Nat.Ac.Science,1996:93:1591-1694
45T.Deschamps.Curve and Shape Extraction with Minimal Path and Level-Sets techniques.Applications to 3D Medical Imaging[R].In Mathématiques Appliquées et Traitements d'Images.Paris,France:Université de Paris Dauphine,2001
46Til Aach,Andre Kaup,et al.Statistical Model-based Change Detection in Moving Video[J].Signal Processing,1993,31(2):165-180
47B.Horn and B.Schunck.Determining optical flow.Artificial Intelligence,1981,17:185-203
48A.N.Tikhonov and V.Y.Arsenin.Solutions of Ill -Posed ProbleMs[J].Wiley,Washington,DC,1977
49A.Bruhn,J.Weickert,and C.Schn"orr.Lucas-Kanade meets Horn-Schunck:Combining local and global optic flow methods[J].International Journal of Computer Vision,2005,61(3):211-231,
50钱东海,张建明.五点约束最小二乘法估计光流速度场[J].计算机辅助设计与图形学学报,2004,16(9):2-4
51Joachim Weickert,Andr_es Bruhn,Thomas Brox and Nils Papenberg.A Survey on Variational Optic Flow Methods for Small Displacements[J].O.Scherzer(Ed.)Mathematical Models for Registration and Applications to Medical Imaging,Berlin,Springer,Mathematics in Industry,Vol.10,2006
52桂本烨,钱徽,朱淼良.梯度计算的改进HS光流算法[J].中国图象图形学报,,2005,10(8):2-6
53杨波,徐光祜.基于纹理约束和参数化运动模型的光流估计[J].中国图象图形学报,2004,9(6):2-4
54陈震,高满屯,沈允文.图象光流场计算技术研究进展[J].中国图象图形学报,2002,7(5):2-4
55陈震,高满屯,沈允文.一种基丁光流场重建三维运动和结构的新方法[J].中国图象图形学报,2003,8(6):3-5
56J.L.Barron,D.J.Fleet,and S.S.Beauchemin.Performance of optical flow techniques.International Journal of Computer Vision,1994,12(1):43-77
2Meier T,Ngan K N.Automatic Segmentation of Moving Objects for Video Object Plane Generation[J].IEEE Trans.Circuits System for Video Technology,1998,8(5):525-538
3邓玉春,姜昱明,张建荣.视频序列图像中运动对象分割综述.计算机应用研究,2005,1:8-11
4B KP Horn,B G Schunck.Determining Optical Flow[J].Artificial Intelligence,1981,17:185-203
5B O Lucas,Z Kanade.An IterativeImage Registration Technique with an Application to Stereo Vision[C].Processing of the International Joint Conference on Artificial Intelligence,1981
6H-H Nagel.Displacement Vectors Derived from Second-Order Intensity Variations in Image Sequence[J].Computer Graphics and Image Processing,1983,21:299-324
7M.J.Black and P.Anandan.Robust dynamic motion estimation over time[J].In Proc.1991 IEEE Computer SocietyConference on Computer Vision and PatternRecognition,Maul,HI,1991,292-302
8E.Memin and P.Perez.Dense estimation and object-based segmentation of the optical flow with robust techniques[J].IEEE Transactions on Image Processing,1998,7(5):703-719
9C.Schnorr.On functionals with greyvalue-controlled smoothness terms for determining optical flow[J].IEEE Transactions on PatternAnalysis and kIachine Intelligence,1993,15:1074-1079
10H.-H.Nagel and W.Enkelmann.An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,1986,8:565-593
11L.Alvarez,J.Weickert,and J.Sanchez.Reliable estimation of dense optical flow fields with large displacements[J].International JournalofComputerVision,2000,39(1):41-56
12H.-H.Nagel.Constraints for the estimation of displacement vector fields from image sequences[J].In Proc.Eighth International Joint Conference on Artificial Intelligence,volume 2,Karlsruhe,West Germany,1983,945-951
13D.Shulman and J.Herve.Regularization of discontinuous flow fields[J].In Proc.Workshop on Visual Motion,Irvine,CA,1989,81-90
14F.Heitz and P.Bouthemy.Multimodal estimation of discontinuous optical flow using Markov random fields[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 1993,15(12):1217-1232
15 P. Nesi. Variational approach to optical flow estimation managing discontinuities [J]. Image and Vision Computing, 1993, 11(7): 419-439
16 J. Weickert and C. Schnorr. A theoretical framework for convex regularizes in PDE-based computation of image motion [J]. International Journal of Computer Vision, 2001, 45(3):245-264
17 H.-H. Nagel. Extending the ' oriented smoothness constraint' into the temporal domain and the estimation of derivatives of optical flow [J]. In O. Faugeras, editor, Computer Vision - ECCV ' 90, volume 427 of Lecture Notes in Computer Science, Springer, Berlin, 1990, 139-148
18 J. Weickert and C. Schnorr. Variational optic flow computation with a spatio-temporal smoothness constraint [J]. Journal of Mathematical Imaging and Vision, 2001, 14(3):245-255
19 E P Ong. Robust Optical Flow Computation Based on Least Median of Squares Regression [J]. Computer Vision, 1999, 31(1): 51-82
20 A B Hadiashar, D Suter. Robust Optical Flow Computation [J]. Computer Vision, 1998, 29(1): 59-77
21 Roland Mech, Michael Wollborn. A Noise Robust Method for 2D Shape Estimation of Moving Objects in Video Sequences Considering a Moving Camera [J]. Signal Processing, 1998, 66(2):203-217
22 Ramesh Jain. Dynamic Scene Analysis Using Pixel-based Processes [J]. Computer, 1981, 14(8): 12-18
23 S N Jayaramamurthy, Ramesh Jain. An Approach to the Segmentation of Textured Dynamic Scenes [J]. Computer Graphics and Image Processing, 1983, 21: 239-261
24 Robert Thoma, Matthias Bierling. Motion Compensating Interpolation Considering Covered and Uncovered Background [J]. Signal Processing: Image Communication, 1989, 1: 191-212
25 Til Aach, Andre Kaup, et al. Statistical Model-based Change Detection in Moving Video [J]. Signal Processing, 1993, 31(2): 165-180
26 A Div G. Determining Three-Dimensional Motion and Structure form Optical Flow Generated by Several Moving Objects [J]. IEEE Trans. on Pattern Analysis Machine Intelligence, 1985, 7(4): 384-401
27 Wang J, Adelson E. Representing Moving Images with Layers [J]. IEEE Trans. on Image Processing, 1994, 3(5): 625-638
28 S Geman, D Geman. Stochastic Relaxation, Gipps Distributions and the Bayesian Restoration of Images [J]. IEEETrans. on Pattern Analysis and Machine Intelligence, 1984, 6(6): 721-741
29 Murray D W, Buxton B F. Scene Segmentation form Visual Motion Using Global Optimization [J]. IEEE Trans. on Pattern Analysis and Machine Intelligence, 1987, 9(2): 220-228
30 Michael M Chang, A Murat Tekalp, M Ibrahim Sezan. An Algorithm for Simultaneous Motion Estimation and Segmentation [C]. IEEE Internal Conference on Acoustics, Speech and Signal Processing (ICASSP' 94), Adelaide, Australia, 1994. 221-224
31 Patrick Bouthemy, Edouard Francois. Motion Segmentation and Qualitative Dynamic Scene Analysis from an Image Sequence [J]. International Journal of Computing Vision, 1993, 10 (2): 157-182
32 S. Osher and J. A. Sethian. Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations[J]. Journal of Computational Physics, 1988: vol. 79, pp. 12-49
33 V. Caselles, F. Catte, T. Coll, and F. Dibos. A geometric model for active contours[J]. Numerische Mathematik, 1993: vol. 66, pp. 1-31
34 R. Malladi, J. A. Sethian, and B. C. Vemuri. Shape modeling with front propagation: A level set approach[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1995: vol. 17, pp. 158-175
35 S. Kichenassamy, A. Kumar, P. Olver, A. Tannenbaum, and A. Yezzi. Gradient flows and geometric active contour models[R]. in the Proceedings of IEEE International Conference in Computer Vision, pp. 810-815, Cambridge, MA, USA, 1995
36 V. Caselles, R. Kimmel, and G. Sapiro. Geodesic active contours[A]. In the Proceedings of International Conference on Computer Vision[C]. Cambridge, MA, USA, 1995: pp. 694-699
37 Zhao H. K., Chan T., Merriman B. and Osher S.. A variational level set approach to multiphase motion. J. Comput. Phys., 1996, 127: 179-195
38 李俊. 基于曲线演化的图像分割方法及应有研究[D]. 上海：上海交通大学,2002.
39 D. Mumford, J. Shah. Boundary detection by minimizing functional [A]. Proc. IEEE Conf. Computer Vision and Pattern Recognition[C]. San Francisco, CA, 1985:188-211
40 F.T. Chan, L. Vese. Active contours without edges[J]. IEEE Transactions on Image Processing, 2001:10(2), 266-277
41 L. A. Vese, T. F. Chan. A Multiphase Level Set Framework for Image Segmentation Using the Mumford and Shah Model [J] . International Journal of Computer Vision, 2002, 50(3): 271-293
42 Comes J and Fangeras O. Reconciling distance function and level sets[J]. Visual Communic and Image Representation, 2000:vol. 11:209-223
43 D. Chop. Computing minimal surfaces via Level Set curvature flow[J]. Journal of Computational Physic,1993:106:77-91
44J.Sethian.A fast marching level set method for monotonically advancing fronts[J].In Proc.Nat.Ac.Science,1996:93:1591-1694
45T.Deschamps.Curve and Shape Extraction with Minimal Path and Level-Sets techniques.Applications to 3D Medical Imaging[R].In Mathématiques Appliquées et Traitements d'Images.Paris,France:Université de Paris Dauphine,2001
46Til Aach,Andre Kaup,et al.Statistical Model-based Change Detection in Moving Video[J].Signal Processing,1993,31(2):165-180
47B.Horn and B.Schunck.Determining optical flow.Artificial Intelligence,1981,17:185-203
48A.N.Tikhonov and V.Y.Arsenin.Solutions of Ill -Posed ProbleMs[J].Wiley,Washington,DC,1977
49A.Bruhn,J.Weickert,and C.Schn"orr.Lucas-Kanade meets Horn-Schunck:Combining local and global optic flow methods[J].International Journal of Computer Vision,2005,61(3):211-231,
50钱东海,张建明.五点约束最小二乘法估计光流速度场[J].计算机辅助设计与图形学学报,2004,16(9):2-4
51Joachim Weickert,Andr_es Bruhn,Thomas Brox and Nils Papenberg.A Survey on Variational Optic Flow Methods for Small Displacements[J].O.Scherzer(Ed.)Mathematical Models for Registration and Applications to Medical Imaging,Berlin,Springer,Mathematics in Industry,Vol.10,2006
52桂本烨,钱徽,朱淼良.梯度计算的改进HS光流算法[J].中国图象图形学报,,2005,10(8):2-6
53杨波,徐光祜.基于纹理约束和参数化运动模型的光流估计[J].中国图象图形学报,2004,9(6):2-4
54陈震,高满屯,沈允文.图象光流场计算技术研究进展[J].中国图象图形学报,2002,7(5):2-4
55陈震,高满屯,沈允文.一种基丁光流场重建三维运动和结构的新方法[J].中国图象图形学报,2003,8(6):3-5
56J.L.Barron,D.J.Fleet,and S.S.Beauchemin.Performance of optical flow techniques.International Journal of Computer Vision,1994,12(1):43-77