运动粒子群三维轨迹获取方法研究
|
摘要
自然场景中存在着很多可以抽象为三维空间中运动粒子群的现象,例如空中飞翔的鸟群、昆虫群以及水中游弋的鱼群等等。研究这些现象对于揭示动物的群体行为规律有着重要的意义。目前对这些现象的研究主要是基于肉眼观察,使得研究停留在理论假设、数学模拟的阶段。若能够得到它们的三维运动轨迹,对揭示这些自然现象背后隐藏的机理有着重要的意义。
为了获取这些粒子状的三维运动群体,我们首先提出了一种基于相对极线运动的方法来求解立体匹配问题。该方法利用相对极线运动的两个原则：1)真正的对应点始终在对方的极线上；2)在矫正过的立体图像中,真正的对应点在垂于极线方向的速度分量始终相等；来重建包含三维粒子群的动态场景。该方法通过二维投影点轨迹匹配来解决立体匹配中晕倒的不确定性问题(一个投影点在另外一个相机平面上有若干个对应点)。并且在该方法的基础之上,我们提出了一种新的动态结构光,该结构光由运动的点组成。实验结果表明,该方法可有效地重建包含三维运动粒子群的动态场景,所提出的动态结构光能够用于获取没有纹理的表面的三维结构。
基于相对极线运动的方法最大的缺点就是强烈依赖于图像平面上投影点轨迹跟踪的性能。若图像平面上跟踪的投影点轨迹是错误的,那么将使三维重建结果产生巨大的误差。注意到轨迹重建问题中,立体匹配与物体跟踪实际上是相互耦合的,我们提出了一种基于全局优化的方法来同时解决这两个问题。该方法将可能对应的一对投影点看成是一个“配对”,把原轨迹重建问题则转化为配对选择的问题,然后通过优化一个代价函数,得到最优的配对序列。最终,粒子的三维运动轨迹从这些配对序列中重建得出。通过实验对比,我们验证了该方法重建粒子群的三维运动轨迹的性能远超现有方法。我们使用该方法成功重建出在玻璃箱中一群高速飞行的果蝇(上百只)的三维运动轨迹,尚属国际首次,对果蝇群体行为以及飞行动力学的研究有着重大的意义。
中图分类号：TP181
Phenomena such as insect swarms, bird flocks, and fish schools occur preva-lently in our environments. They have attracted significant attention by scientists in many disciplines [58,51,4]. The availability of the 3D motion trajectory for each individual in the swarm can greatly facilitate the study of animal collec-tive behavior. In order to reconstruct such 3D scenes made up of large number of particles, we first proposed an approach-Relative Epipolar Motion(REM)-towards solving the correspondence problem in stereopsis by utilizing the motion clue. Using the relative epipolar motion cues, it can reconstruct dynamic scenes of large number of indistinguishable moving objects. It offers an alternative way to project structured lights in active mode for deforming surface reconstruction. The REM method however strongly relies on the performance of tracking on 2D image planes. If tracking failed, it will cause significant reconstruction error. Noticing stereo matching and tracking are in fact coupled, we proposed a global optimiza-tion method that treats a possible corresponding feature points as a'pairing'. The trajectory reconstruction problem becomes a process of pairing selection. Through minimizing a cost function, we obtain the optimum pairing sequences. The trajec-tories are then reconstructed from these sequences by triangulation. Experiments show the significant advantage of the proposed method over existing methods. We used the proposed method to reconstruct the 3D trajectories of about hundreds of fruit flies simultaneously flying in a glass box. We were the first to our best knowledge that obtain complete 3D trajectories of a swarm of hundreds of flying fruit flies, enabling biologists to conduct thorough study of collective behavior of animals.
为了获取这些粒子状的三维运动群体,我们首先提出了一种基于相对极线运动的方法来求解立体匹配问题。该方法利用相对极线运动的两个原则：1)真正的对应点始终在对方的极线上；2)在矫正过的立体图像中,真正的对应点在垂于极线方向的速度分量始终相等；来重建包含三维粒子群的动态场景。该方法通过二维投影点轨迹匹配来解决立体匹配中晕倒的不确定性问题(一个投影点在另外一个相机平面上有若干个对应点)。并且在该方法的基础之上,我们提出了一种新的动态结构光,该结构光由运动的点组成。实验结果表明,该方法可有效地重建包含三维运动粒子群的动态场景,所提出的动态结构光能够用于获取没有纹理的表面的三维结构。
基于相对极线运动的方法最大的缺点就是强烈依赖于图像平面上投影点轨迹跟踪的性能。若图像平面上跟踪的投影点轨迹是错误的,那么将使三维重建结果产生巨大的误差。注意到轨迹重建问题中,立体匹配与物体跟踪实际上是相互耦合的,我们提出了一种基于全局优化的方法来同时解决这两个问题。该方法将可能对应的一对投影点看成是一个“配对”,把原轨迹重建问题则转化为配对选择的问题,然后通过优化一个代价函数,得到最优的配对序列。最终,粒子的三维运动轨迹从这些配对序列中重建得出。通过实验对比,我们验证了该方法重建粒子群的三维运动轨迹的性能远超现有方法。我们使用该方法成功重建出在玻璃箱中一群高速飞行的果蝇(上百只)的三维运动轨迹,尚属国际首次,对果蝇群体行为以及飞行动力学的研究有着重大的意义。
中图分类号：TP181
Phenomena such as insect swarms, bird flocks, and fish schools occur preva-lently in our environments. They have attracted significant attention by scientists in many disciplines [58,51,4]. The availability of the 3D motion trajectory for each individual in the swarm can greatly facilitate the study of animal collec-tive behavior. In order to reconstruct such 3D scenes made up of large number of particles, we first proposed an approach-Relative Epipolar Motion(REM)-towards solving the correspondence problem in stereopsis by utilizing the motion clue. Using the relative epipolar motion cues, it can reconstruct dynamic scenes of large number of indistinguishable moving objects. It offers an alternative way to project structured lights in active mode for deforming surface reconstruction. The REM method however strongly relies on the performance of tracking on 2D image planes. If tracking failed, it will cause significant reconstruction error. Noticing stereo matching and tracking are in fact coupled, we proposed a global optimiza-tion method that treats a possible corresponding feature points as a'pairing'. The trajectory reconstruction problem becomes a process of pairing selection. Through minimizing a cost function, we obtain the optimum pairing sequences. The trajec-tories are then reconstructed from these sequences by triangulation. Experiments show the significant advantage of the proposed method over existing methods. We used the proposed method to reconstruct the 3D trajectories of about hundreds of fruit flies simultaneously flying in a glass box. We were the first to our best knowledge that obtain complete 3D trajectories of a swarm of hundreds of flying fruit flies, enabling biologists to conduct thorough study of collective behavior of animals.
引文
[1]Y. Bar-Shalom. T. Fortmann. and M. Scheffe. Joint probabilistic data associ-ation for multiple targets in clutter. In Proc. Conf. on Information Sciences and Systems, pages 404-409,1980.
[2]M. Bertero, T.A. Poggio, and V. Torre. Ill-posed problems in early vision. Proceedings of the IEEE,76(8):869-889,1988.
[3]M. Betke, DE Hirsh, A. Bagchi, NI Hristov, NC Makris, and TH Kunz. Track-ing large variable numbers of objects in clutter. In IEEE Conference on Computer Vision and Pattern Recognition, pages 1-8.2007.
[4]M. Betke, D.E. Hirsh, N.C. Makris, G.F. McCracken, M. Procopio, N.I. Hris-tov, S. Tang. A. Bagchi. J.D. Reichard. J.W. Horn, et al. Thermal imaging reveals significantly smaller brazilian free-tailed bat colonies than previously estimated. Journal of Mammalogy,89(1):18-24,2008.
[5]C.M. Bishop et al. Pattern recognition and machine learning. Springer New York:,2006.
[6]J. Black. T. Ellis, and P. Rosin. Multi view image surveillance and tracking. In IEEE Workshop on Motion and Video Computing, volume 6,2002.
[7]M.J. Black and A. Rangarajan. On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. International Journal of Computer Vision,19(1):57-91,1996.
[8]A. Blake and A. Zisserman. Visual reconstruction. MIT press Cambridge, MA,1987.
[9]K. Branson and S. Belongie. Tracking multiple mouse contours (without too many samples). In IEEE Computer Society Conference on Computer Vision and Pattern Recognition,2005. CVPR 2005, volume 1,2005.
[10]K. Branson, A.A. Robie, J. Bender, P. Perona. and M.H. Dickinson. High-throughput ethomics in large groups of Drosophila. Nature methods,6(6):451-457.2009.
[11]C. Bregler and J. Malik. Tracking people with twists and exponential maps. In IEEE Computer Society Conference on Computer Vision and Pattern Recog-nition, pages 8-15. Citeseer,1998.
[12]T. Brodski, C. Fermuller, and Y. Aloimonos. Shape from Video. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 2, pages 1-151. Citeseer.1999.
[13]O. Cappe, SJ Godsill. and E. Moulines. An overview of existing methods and recent advances in sequential Monte Carlo. Proceedings of the IEEE, 95(5):899-924,2007.
[14]Editorial comments. No fruit fly an island. Nature methods,6(4):395-395, 2009.
[15]IJ Cox and SL Hingorani. An efficient implementation of reid's multiple hypothesis trackingalgorithm and its evaluation for the purpose of visual tracking. IEEE Transactions on pattern analysis and machine intelligence, 18(2):138-150,1996.
[16]H.J. Dahmen and J. Zeil. Recording and reconstructing three-dimensional trajectories:a versatile method for the field biologist. Proceedings of the Royal Society of London. Series B. Biological Sciences.222(1226):107-113, 1984.
[17]R. Danchick and GE Newnam. A fast method for finding the exact n-best hypotheses formultitarget tracking. IEEE Transactions on Aerospace and Electronic Systems,29(2):555-560.1993.
[18]H. Dankert. L. Wang, E.D. Hoopfer, D.J. Anderson, and P. Perona. Au-tomated monitoring and analysis of social behavior in Drosophila. Nature methods,6(4):297-303,2009.
[19]J. Davis. D. Nehab, R. Ramamoorthi, and S. Rusinkiewicz. Spacetime stereo: A unifying framework for depth from triangulation. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 296-302.2005.
[20]R. De Nardi, O. Holland, J. Woods, and A. Clark. SwarMAV:A swarm of miniature aerial vehicles. In Proceedings of the 21st Bristol International UAV Systems Conference. Citeseer,2006.
[21]S.L. Dockstader and A.M. Tekalp. Multiple camera tracking of interacting and occluded human motion. Proceedings of the IEEE,89(10):1441-1455, 2001.
[22]H. Du, D. Zou, and Y.Q. Chen. Relative epipolar motion of tracked features for correspondence in binocular stereo. In IEEE 11th International Confer-ence on Computer Vision,2007.
[23]H. Du, D. Zou, and Y.Q. Chen. Relative Epipolar Motion of Tracked Fea-tures for Correspondence in Binocular Stereo. In IEEE 11th International Conference on Computer Vision,2007. ICCV 2007,2007.
[24]K. Egemen Ozden. K. Cornelis, L. Van Eycken, and L. Van Gool. Reconstruct-ing 3D trajectories of independently moving objects using generic constraints. Computer Vision and Image Understanding,96(3):453-471,2004.
[25]D. Engelmann, C. Garbe, M. St "ohr, P. Geissler, F. Hering, and B. J "ahne. Stereo particle tracking. In Proceedings of the 8th International Sym-posium on Flow Visualisation, Sorrento,Italy, September, pages 1-4,1998.
[26]D. Engelmann, C. Garbe, and M.G. Stohr. Stereo particle tracking. In Proc. of 8th Int. Symp. on Flow Visualization (CD-ROM), pages 240-1,1998.
[27]D.A. Forsyth and J. Ponce. Computer Vision:A Modern Approach. Prentice Hall 2002.
[28]S.N. Fry, R. Sayaman. and M.H. Dickinson. The aerodynamics of free-flight maneuvers in Drosophila. Science,300(5618):495,2003.
[29]A. Fusiello, E. Trucco,and A. Verri. A compact algorithm for rectification of stereo pairs. Machine Vision and Applications,12(1):16-22,2000.
[30]Y. Ge, S.S. Cha, and J.H. Park. Study of particle tracking algorithms based on neural networks for stereoscopic tracking velocimetry. Optics and Lasers in Engineering,44(6):623-636,2006.
[31]S. GEMAN and D. GEMAN. Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence,6(6):721-741,1984.
[32]S. Geman, D. Geman, K. Abend, TJ Harley. and LN Kanal. Stochastic relax-ation, Gibbs distributions and the Bayesian restoration of images*. Journal of Applied Statistics,20(5):25-62,1993.
[33]G.H. Golub and C.F. Van Loan. Matrix computations. Johns Hopkins Univ Pr,1996.
[34]D. Grover. J. Tower, and S. Tavare.O fly, where art thou? Journal of The Royal Society Interface,5(27):1181.2008.
[35]YG Guezennec, RS Brodkey, N. Trigui, and JC Kent. Algorithms for fully automated three-dimensional particle tracking velocimetry. Experiments in Fluids,17(4):209-219, 1994.
[36]I. Guskov, S. Klibanov, and B. Bryant. Trackable surfaces. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer ani-mation, pages 251-257. Eurographics Association Aire-la-Ville, Switzerland, Switzerland,2003.
[37]J. Hadamard. Lectures on Cauchy's problem in linear partial differential equa-tions. Dover Pubns.2003.
[38]M. J. Hannah. Computer matching of areas in stereo images.1974.
[39]R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press,2003.
[40]W. Hu, M. Hu, X. Zhou, T. Tan, J. Lou, and S. Maybank. Principal axis-based correspondence between multiple cameras for people tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence,28(4):663,2006.
[41]C. Hue, J.P. Le Cadre, P. Perez, and R.I. IRISA. Tracking multiple objects with particle filtering. IEEE Transactions on Aerospace and Electronic Sys-tems,38(3):791-812,2002.
[42]S.J. Julier and J.K. Uhlmann. A new extension of the Kalman filter to nonlinear systems. In Int. Symp. Aerospace/Defense Sensing. Simul. and Controls, volume 3, page 26. Citeseer,1997.
[43]R.E. Kalman. A new approach to linear filtering and prediction problems. Journal of basic Engineering,82(1):35-45:1960.
[44]RE Kalman and RS Bucy. New results in linear filtering and prediction theory. Kalman Filtering:Theory and Application, page 34.
[45]N. KASAGI and K. NISHINO. Probing turbulence with three-dimensional particle-tracking velocimetry. Experimental thermal and fluid science, 4(5):601-612,1991.
[46]Z. Khan, T. Balch, and F. Dellaert. Mcmc-based particle filtering for track-ing a variable number of interacting targets. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1805-1918,2005.
[47]G. Kitagawa. Non-Gaussian state-space modeling of nonstationary time se-ries. Journal of the American statistical association, pages 1032-1041.1987.
[48]V. Kolmogorov and R. Zabih. Computing visual correspondence with oc-clusions via graph cuts. In International Conference on Computer Vision, volume 2. pages 508-515. Citeseer.2001.
[49]Y. Liu and K.M. Passino. Swarm intelligence:Literature overview. Dept. of Electrical Engineering, The Ohio State University,2015.
[50]HG Maas, A. Gruen, and D. Papantoniou. Particle tracking velocimetry in three-dimensional flows. Experiments in Fluids,15(2):133-146,1993.
[51]Gaby Maimon, Andrew D. Straw, and Michael H. Dickinson. A simple vision-based algorithm for decision making in flying drosophila. Current Biology, 18(6):464-470; March 2008.
[52]P.F. Major and L.M. Dill. The three-dimensional structure of airborne bird flocks. Behavioral Ecology and Sociobiology,4(2):111-122,1978.
[53]NA Malik, T. Dracos, and DA Papantoniou. Particle tracking velocimetry in three-dimensional flows-part ii:particle tracking. Experiments in Fluids, 15(4):279-294; 1993.
[54]J. Marroquin, S. Mitter, and T. Poggio. Probabilistic solution of ill-posed problems in computational vision. Journal of the American Statistical Asso-ciation,82(397):76-89:1987.
[55]K.G. Murty. An algorithm for ranking all the assignments in order of increas-ing cost. Operations Research,16(3):682-687.1968.
[56]E. Muybridge. Animals in motion. Dover Pubns.1957.
[57]S.K. Nayar, M. Watanabe. and M. Noguchi. Real-time focus range sen-sor. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(12):1186-1198,1996.
[58]KS Norris and CR Schilt. Cooperative societies in three-dimensional space: on the origins of aggregations, flocks, and schools, with special reference to dolphins and fish. Ethology and Sociobiology,9(2-4):149-179,1988.
[59]S. Oh. S. Russell, and S. Sastry. Markov chain Monte Carlo data association for general multiple-target tracking problems. In 43rd IEEE Conference on Decision and Control,2004. CDC, volume 1,2004.
Y. Ohta and T. Kanade. Stereo by intra-and inter-scanline search using dy-namic programming. IEEE Transactions on pattern analysis and machine intelligence,7(2):139-154,1985.
[61]C.H. Papadimitriou and K. Steiglitz. Combinatorial optimization:algorithms and complexity. Dover Pubns,1998.
[62]D. Papantoniou and T. Dracos. Analyzing 3-Dimensional Turbulent Motions in Open Channel Flow by Use of Stereoscopy and Particle Tracking. Advances in Turbulence 2. Springer. Heidelberg,1989.
[63]M. Pascoal, M.E. Captivo. and J. Climaco. A note on a new variant of Murty's ranking assignments algorithm.4OR:A Quarterly Journal of Op-erations Research, 1(3):243-255; 2003.
[64]C.R. Pedersen. L. Relund Nielsen, and K.A. Andersen. An algorithm for rank-ing assignments using reoptimization. Computers and Operations Research, 35(11):3714-3726,2008.
[65]F. Pereira, H. Stuer, E.C Graff, and M. Gharib. Two-frame 3d particle track-ing. Measurement Science and Technology,17(7):1680-1692,2006.
[66]TJ Pitcher. A periscopic method for determining the three-dimensional po-sitions of fish in schools. J. Fish. Res. Board Can,32:1533-1538,1975.
[67]C. Poelman and T. Kanade. A paraperspective factorization method for shape and motion recovery. Computer Vision-ECCV'94, pages 97-108.
[68]T. Poggio, V. Torre, and C. Koch. Computational vision and regularization theory. Image Understanding,3:1-18,1989.
[69]D. Pritchard and W. Heidrich. Cloth motion capture. In International Con-ference on Computer Graphics and Interactive Techniques, pages 1-1. ACM New York, NY, USA,2003.
[70]D. Reid. An algorithm for tracking multiple targets. IEEE Transactions on Automatic Control,24(6):843-854,1979.
[71]J. Kosinski Robert. A Literature Review on Reaction Time. http://biae. clemson. edu/bpc/bp/Lab/110/reaction.htm.
[72]S. Roy and I.J. Cox. A maximum-flow formulation of the n-camera stereo correspondence problem.1998.
[73]TW Ryan, RT Gray, and BR Hunt. Prediction of correlation errors in stereo-pair images. Optical Engineering,19(3):312-322,1980.
[74]Y. Sata and N. Kasagi. Improvement toward high measurement resolution in three-dimensional particle tracking velocimetry. Flow visualization Ⅵ, pages 792-796,1992.
[75]D. Scharstein and R. Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International journal of computer vision, 47(1):7-42,2002.
[76]R. Szeliski and S.B. Kang. Recovering 3D Shape and Motion from Image Streams Using Nonlinear Least Squares. Journal of Visual Communication and Image Representation,5(1):10-28,1994.
[77]L.F. Tammero and M.H. Dickinson. The influence of visual landscape on the free flight behavior of the fruit fly drosophila melanogaster. Journal of Experimental Biology,205(3):327-343,2002.
[78]D. Terzopoulos. Regularization of inverse visual problems involving discon-tinuities. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(4):413-424,1986.
[79]C. Tomasi and T. Kanade. Shape and motion from image streams under orthography:a factorization method. International Journal of Computer Vision,9(2):137-154,1992.
[80]C.J. Veenman, M.J.T. Reinders, and E. Backer. Resolving motion correspon-dence for densely moving points. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 54-72,2001.
[81]A. Veeraraghavan, R. Chellappa, and M. Srinivasan. Shape-and-behavior en-coded tracking of bee dances. IEEE Transactions on Pattern Analysis and Machine Intelligence,30(3):463-476,2008.
[82]O. Veksler. Stereo matching by compact windows via minimum ratio cycle. In Eighth International Conference on Computer Vision (ICCV 2001), pages 540-547. Citeseer,2001.
[83]O. Veksler and R. Zabih. Efficient graph-based energy minimization methods in computer vision. Cornell University, Ithaca, NY,1999.
[84]G. Welch and G. Bishop. An introduction to the Kalman filter. University of North Carolina at Chapel Hill, Chapel Hill, NC,1995.
L. Williams. Performance-driven facial animation. ACM SIGGRAPH Com-puter Graphics,24(4):235-242,1990.
[86]A. Yilmaz,0. Javed, and M. Shah. Object tracking:A survey. ACM Com-puting Surveys (CSUR),38(4):13,2006.
[87]Q. Yu, G. Medioni, and I. Cohen. Multiple target tracking using spatio-temporal markov chain monte carlo data association. In IEEE Conference on Computer Vision and Pattern Recognition,2007. CVPR'07, pages 1-8,2007.
[88]L. Zhang, B. Curless, and S. Seitz. Spacetime stereo:Shape recovery for dynamic scenes. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 2. Citeseer,2003.
[89]L. Zhang, B. Curless, and S.M. Seitz. Rapid shape acquisition using color structured light and multi-pass dynamic programming. In The 1st IEEE International Symposium on 3D Data Processing, Visualization, and Trans-mission, pages 24-36. Citeseer,2002.
[90]R. Zhang, P.S. Tsai, JE Cryer, and M. Shah. Shape-from-shading:a survey. IEEE transactions on pattern analysis and machine intelligence,21(8):690-706,1999.
[91]S. Zhang and P. Huang. High-resolution, real-time 3D shape acquisition. In Computer Vision and Pattern Recognition Workshop,2004 Conference on, pages 28-28,2004.
[92]Z. Zhang. A flexible new technique for camera calibration. IEEE Transactions, on Pattern Analysis and Machine Intelligence,22(11),2000.
[93]Q. Zhao and Y.Q. Chen. High Precision Synchronization Of Video Cam-eras Using A Single Binary Light Source. Journal of Electronic Imaging, 18(4):040501-1,2009.
[94]CL Zitnick and T. Kanade. A cooperative algorithm for stereo matching and occlusion detection. IEEE Transactions on Pattern Analysis and Machine Intelligence,22(7):675-684,2000.
[95]D. Zou, Q. Zhao, H.S. Wu, and Y.Q. Chen. Reconstructing 3d motion tra-jectories of particle swarms by global correspondence selection. In IEEE 12th International Conference on Computer Vision,2009.
[2]M. Bertero, T.A. Poggio, and V. Torre. Ill-posed problems in early vision. Proceedings of the IEEE,76(8):869-889,1988.
[3]M. Betke, DE Hirsh, A. Bagchi, NI Hristov, NC Makris, and TH Kunz. Track-ing large variable numbers of objects in clutter. In IEEE Conference on Computer Vision and Pattern Recognition, pages 1-8.2007.
[4]M. Betke, D.E. Hirsh, N.C. Makris, G.F. McCracken, M. Procopio, N.I. Hris-tov, S. Tang. A. Bagchi. J.D. Reichard. J.W. Horn, et al. Thermal imaging reveals significantly smaller brazilian free-tailed bat colonies than previously estimated. Journal of Mammalogy,89(1):18-24,2008.
[5]C.M. Bishop et al. Pattern recognition and machine learning. Springer New York:,2006.
[6]J. Black. T. Ellis, and P. Rosin. Multi view image surveillance and tracking. In IEEE Workshop on Motion and Video Computing, volume 6,2002.
[7]M.J. Black and A. Rangarajan. On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. International Journal of Computer Vision,19(1):57-91,1996.
[8]A. Blake and A. Zisserman. Visual reconstruction. MIT press Cambridge, MA,1987.
[9]K. Branson and S. Belongie. Tracking multiple mouse contours (without too many samples). In IEEE Computer Society Conference on Computer Vision and Pattern Recognition,2005. CVPR 2005, volume 1,2005.
[10]K. Branson, A.A. Robie, J. Bender, P. Perona. and M.H. Dickinson. High-throughput ethomics in large groups of Drosophila. Nature methods,6(6):451-457.2009.
[11]C. Bregler and J. Malik. Tracking people with twists and exponential maps. In IEEE Computer Society Conference on Computer Vision and Pattern Recog-nition, pages 8-15. Citeseer,1998.
[12]T. Brodski, C. Fermuller, and Y. Aloimonos. Shape from Video. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 2, pages 1-151. Citeseer.1999.
[13]O. Cappe, SJ Godsill. and E. Moulines. An overview of existing methods and recent advances in sequential Monte Carlo. Proceedings of the IEEE, 95(5):899-924,2007.
[14]Editorial comments. No fruit fly an island. Nature methods,6(4):395-395, 2009.
[15]IJ Cox and SL Hingorani. An efficient implementation of reid's multiple hypothesis trackingalgorithm and its evaluation for the purpose of visual tracking. IEEE Transactions on pattern analysis and machine intelligence, 18(2):138-150,1996.
[16]H.J. Dahmen and J. Zeil. Recording and reconstructing three-dimensional trajectories:a versatile method for the field biologist. Proceedings of the Royal Society of London. Series B. Biological Sciences.222(1226):107-113, 1984.
[17]R. Danchick and GE Newnam. A fast method for finding the exact n-best hypotheses formultitarget tracking. IEEE Transactions on Aerospace and Electronic Systems,29(2):555-560.1993.
[18]H. Dankert. L. Wang, E.D. Hoopfer, D.J. Anderson, and P. Perona. Au-tomated monitoring and analysis of social behavior in Drosophila. Nature methods,6(4):297-303,2009.
[19]J. Davis. D. Nehab, R. Ramamoorthi, and S. Rusinkiewicz. Spacetime stereo: A unifying framework for depth from triangulation. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 296-302.2005.
[20]R. De Nardi, O. Holland, J. Woods, and A. Clark. SwarMAV:A swarm of miniature aerial vehicles. In Proceedings of the 21st Bristol International UAV Systems Conference. Citeseer,2006.
[21]S.L. Dockstader and A.M. Tekalp. Multiple camera tracking of interacting and occluded human motion. Proceedings of the IEEE,89(10):1441-1455, 2001.
[22]H. Du, D. Zou, and Y.Q. Chen. Relative epipolar motion of tracked features for correspondence in binocular stereo. In IEEE 11th International Confer-ence on Computer Vision,2007.
[23]H. Du, D. Zou, and Y.Q. Chen. Relative Epipolar Motion of Tracked Fea-tures for Correspondence in Binocular Stereo. In IEEE 11th International Conference on Computer Vision,2007. ICCV 2007,2007.
[24]K. Egemen Ozden. K. Cornelis, L. Van Eycken, and L. Van Gool. Reconstruct-ing 3D trajectories of independently moving objects using generic constraints. Computer Vision and Image Understanding,96(3):453-471,2004.
[25]D. Engelmann, C. Garbe, M. St "ohr, P. Geissler, F. Hering, and B. J "ahne. Stereo particle tracking. In Proceedings of the 8th International Sym-posium on Flow Visualisation, Sorrento,Italy, September, pages 1-4,1998.
[26]D. Engelmann, C. Garbe, and M.G. Stohr. Stereo particle tracking. In Proc. of 8th Int. Symp. on Flow Visualization (CD-ROM), pages 240-1,1998.
[27]D.A. Forsyth and J. Ponce. Computer Vision:A Modern Approach. Prentice Hall 2002.
[28]S.N. Fry, R. Sayaman. and M.H. Dickinson. The aerodynamics of free-flight maneuvers in Drosophila. Science,300(5618):495,2003.
[29]A. Fusiello, E. Trucco,and A. Verri. A compact algorithm for rectification of stereo pairs. Machine Vision and Applications,12(1):16-22,2000.
[30]Y. Ge, S.S. Cha, and J.H. Park. Study of particle tracking algorithms based on neural networks for stereoscopic tracking velocimetry. Optics and Lasers in Engineering,44(6):623-636,2006.
[31]S. GEMAN and D. GEMAN. Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence,6(6):721-741,1984.
[32]S. Geman, D. Geman, K. Abend, TJ Harley. and LN Kanal. Stochastic relax-ation, Gibbs distributions and the Bayesian restoration of images*. Journal of Applied Statistics,20(5):25-62,1993.
[33]G.H. Golub and C.F. Van Loan. Matrix computations. Johns Hopkins Univ Pr,1996.
[34]D. Grover. J. Tower, and S. Tavare.O fly, where art thou? Journal of The Royal Society Interface,5(27):1181.2008.
[35]YG Guezennec, RS Brodkey, N. Trigui, and JC Kent. Algorithms for fully automated three-dimensional particle tracking velocimetry. Experiments in Fluids,17(4):209-219, 1994.
[36]I. Guskov, S. Klibanov, and B. Bryant. Trackable surfaces. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer ani-mation, pages 251-257. Eurographics Association Aire-la-Ville, Switzerland, Switzerland,2003.
[37]J. Hadamard. Lectures on Cauchy's problem in linear partial differential equa-tions. Dover Pubns.2003.
[38]M. J. Hannah. Computer matching of areas in stereo images.1974.
[39]R. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press,2003.
[40]W. Hu, M. Hu, X. Zhou, T. Tan, J. Lou, and S. Maybank. Principal axis-based correspondence between multiple cameras for people tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence,28(4):663,2006.
[41]C. Hue, J.P. Le Cadre, P. Perez, and R.I. IRISA. Tracking multiple objects with particle filtering. IEEE Transactions on Aerospace and Electronic Sys-tems,38(3):791-812,2002.
[42]S.J. Julier and J.K. Uhlmann. A new extension of the Kalman filter to nonlinear systems. In Int. Symp. Aerospace/Defense Sensing. Simul. and Controls, volume 3, page 26. Citeseer,1997.
[43]R.E. Kalman. A new approach to linear filtering and prediction problems. Journal of basic Engineering,82(1):35-45:1960.
[44]RE Kalman and RS Bucy. New results in linear filtering and prediction theory. Kalman Filtering:Theory and Application, page 34.
[45]N. KASAGI and K. NISHINO. Probing turbulence with three-dimensional particle-tracking velocimetry. Experimental thermal and fluid science, 4(5):601-612,1991.
[46]Z. Khan, T. Balch, and F. Dellaert. Mcmc-based particle filtering for track-ing a variable number of interacting targets. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 1805-1918,2005.
[47]G. Kitagawa. Non-Gaussian state-space modeling of nonstationary time se-ries. Journal of the American statistical association, pages 1032-1041.1987.
[48]V. Kolmogorov and R. Zabih. Computing visual correspondence with oc-clusions via graph cuts. In International Conference on Computer Vision, volume 2. pages 508-515. Citeseer.2001.
[49]Y. Liu and K.M. Passino. Swarm intelligence:Literature overview. Dept. of Electrical Engineering, The Ohio State University,2015.
[50]HG Maas, A. Gruen, and D. Papantoniou. Particle tracking velocimetry in three-dimensional flows. Experiments in Fluids,15(2):133-146,1993.
[51]Gaby Maimon, Andrew D. Straw, and Michael H. Dickinson. A simple vision-based algorithm for decision making in flying drosophila. Current Biology, 18(6):464-470; March 2008.
[52]P.F. Major and L.M. Dill. The three-dimensional structure of airborne bird flocks. Behavioral Ecology and Sociobiology,4(2):111-122,1978.
[53]NA Malik, T. Dracos, and DA Papantoniou. Particle tracking velocimetry in three-dimensional flows-part ii:particle tracking. Experiments in Fluids, 15(4):279-294; 1993.
[54]J. Marroquin, S. Mitter, and T. Poggio. Probabilistic solution of ill-posed problems in computational vision. Journal of the American Statistical Asso-ciation,82(397):76-89:1987.
[55]K.G. Murty. An algorithm for ranking all the assignments in order of increas-ing cost. Operations Research,16(3):682-687.1968.
[56]E. Muybridge. Animals in motion. Dover Pubns.1957.
[57]S.K. Nayar, M. Watanabe. and M. Noguchi. Real-time focus range sen-sor. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(12):1186-1198,1996.
[58]KS Norris and CR Schilt. Cooperative societies in three-dimensional space: on the origins of aggregations, flocks, and schools, with special reference to dolphins and fish. Ethology and Sociobiology,9(2-4):149-179,1988.
[59]S. Oh. S. Russell, and S. Sastry. Markov chain Monte Carlo data association for general multiple-target tracking problems. In 43rd IEEE Conference on Decision and Control,2004. CDC, volume 1,2004.
Y. Ohta and T. Kanade. Stereo by intra-and inter-scanline search using dy-namic programming. IEEE Transactions on pattern analysis and machine intelligence,7(2):139-154,1985.
[61]C.H. Papadimitriou and K. Steiglitz. Combinatorial optimization:algorithms and complexity. Dover Pubns,1998.
[62]D. Papantoniou and T. Dracos. Analyzing 3-Dimensional Turbulent Motions in Open Channel Flow by Use of Stereoscopy and Particle Tracking. Advances in Turbulence 2. Springer. Heidelberg,1989.
[63]M. Pascoal, M.E. Captivo. and J. Climaco. A note on a new variant of Murty's ranking assignments algorithm.4OR:A Quarterly Journal of Op-erations Research, 1(3):243-255; 2003.
[64]C.R. Pedersen. L. Relund Nielsen, and K.A. Andersen. An algorithm for rank-ing assignments using reoptimization. Computers and Operations Research, 35(11):3714-3726,2008.
[65]F. Pereira, H. Stuer, E.C Graff, and M. Gharib. Two-frame 3d particle track-ing. Measurement Science and Technology,17(7):1680-1692,2006.
[66]TJ Pitcher. A periscopic method for determining the three-dimensional po-sitions of fish in schools. J. Fish. Res. Board Can,32:1533-1538,1975.
[67]C. Poelman and T. Kanade. A paraperspective factorization method for shape and motion recovery. Computer Vision-ECCV'94, pages 97-108.
[68]T. Poggio, V. Torre, and C. Koch. Computational vision and regularization theory. Image Understanding,3:1-18,1989.
[69]D. Pritchard and W. Heidrich. Cloth motion capture. In International Con-ference on Computer Graphics and Interactive Techniques, pages 1-1. ACM New York, NY, USA,2003.
[70]D. Reid. An algorithm for tracking multiple targets. IEEE Transactions on Automatic Control,24(6):843-854,1979.
[71]J. Kosinski Robert. A Literature Review on Reaction Time. http://biae. clemson. edu/bpc/bp/Lab/110/reaction.htm.
[72]S. Roy and I.J. Cox. A maximum-flow formulation of the n-camera stereo correspondence problem.1998.
[73]TW Ryan, RT Gray, and BR Hunt. Prediction of correlation errors in stereo-pair images. Optical Engineering,19(3):312-322,1980.
[74]Y. Sata and N. Kasagi. Improvement toward high measurement resolution in three-dimensional particle tracking velocimetry. Flow visualization Ⅵ, pages 792-796,1992.
[75]D. Scharstein and R. Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International journal of computer vision, 47(1):7-42,2002.
[76]R. Szeliski and S.B. Kang. Recovering 3D Shape and Motion from Image Streams Using Nonlinear Least Squares. Journal of Visual Communication and Image Representation,5(1):10-28,1994.
[77]L.F. Tammero and M.H. Dickinson. The influence of visual landscape on the free flight behavior of the fruit fly drosophila melanogaster. Journal of Experimental Biology,205(3):327-343,2002.
[78]D. Terzopoulos. Regularization of inverse visual problems involving discon-tinuities. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(4):413-424,1986.
[79]C. Tomasi and T. Kanade. Shape and motion from image streams under orthography:a factorization method. International Journal of Computer Vision,9(2):137-154,1992.
[80]C.J. Veenman, M.J.T. Reinders, and E. Backer. Resolving motion correspon-dence for densely moving points. IEEE Transactions on Pattern Analysis and Machine Intelligence, pages 54-72,2001.
[81]A. Veeraraghavan, R. Chellappa, and M. Srinivasan. Shape-and-behavior en-coded tracking of bee dances. IEEE Transactions on Pattern Analysis and Machine Intelligence,30(3):463-476,2008.
[82]O. Veksler. Stereo matching by compact windows via minimum ratio cycle. In Eighth International Conference on Computer Vision (ICCV 2001), pages 540-547. Citeseer,2001.
[83]O. Veksler and R. Zabih. Efficient graph-based energy minimization methods in computer vision. Cornell University, Ithaca, NY,1999.
[84]G. Welch and G. Bishop. An introduction to the Kalman filter. University of North Carolina at Chapel Hill, Chapel Hill, NC,1995.
L. Williams. Performance-driven facial animation. ACM SIGGRAPH Com-puter Graphics,24(4):235-242,1990.
[86]A. Yilmaz,0. Javed, and M. Shah. Object tracking:A survey. ACM Com-puting Surveys (CSUR),38(4):13,2006.
[87]Q. Yu, G. Medioni, and I. Cohen. Multiple target tracking using spatio-temporal markov chain monte carlo data association. In IEEE Conference on Computer Vision and Pattern Recognition,2007. CVPR'07, pages 1-8,2007.
[88]L. Zhang, B. Curless, and S. Seitz. Spacetime stereo:Shape recovery for dynamic scenes. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, volume 2. Citeseer,2003.
[89]L. Zhang, B. Curless, and S.M. Seitz. Rapid shape acquisition using color structured light and multi-pass dynamic programming. In The 1st IEEE International Symposium on 3D Data Processing, Visualization, and Trans-mission, pages 24-36. Citeseer,2002.
[90]R. Zhang, P.S. Tsai, JE Cryer, and M. Shah. Shape-from-shading:a survey. IEEE transactions on pattern analysis and machine intelligence,21(8):690-706,1999.
[91]S. Zhang and P. Huang. High-resolution, real-time 3D shape acquisition. In Computer Vision and Pattern Recognition Workshop,2004 Conference on, pages 28-28,2004.
[92]Z. Zhang. A flexible new technique for camera calibration. IEEE Transactions, on Pattern Analysis and Machine Intelligence,22(11),2000.
[93]Q. Zhao and Y.Q. Chen. High Precision Synchronization Of Video Cam-eras Using A Single Binary Light Source. Journal of Electronic Imaging, 18(4):040501-1,2009.
[94]CL Zitnick and T. Kanade. A cooperative algorithm for stereo matching and occlusion detection. IEEE Transactions on Pattern Analysis and Machine Intelligence,22(7):675-684,2000.
[95]D. Zou, Q. Zhao, H.S. Wu, and Y.Q. Chen. Reconstructing 3d motion tra-jectories of particle swarms by global correspondence selection. In IEEE 12th International Conference on Computer Vision,2009.