基于视频的大鼠肢体运动分析
|
摘要
实验动物行为分析是高级神经中枢功能研究的一项重要技术手段,不仅广泛应用于神经心理学、神经药理学等传统领域,而且延伸到人工智能、仿生学、脑机接口等新兴交叉研究领域。肢体运动行为参数化描述是精细行为分析的基础,因此建立动物肢体运动行为自动分析方法对促进相关学科的发展具有重要意义。
实验大鼠的肢体运动具有目标小,运动变化快的特点,对肢体运动检测与分析带来了极大挑战。本文从精确性、稳定性和实时性出发,对实验装置设计、实验方法以及相关图像处理分析算法进行了研究。设计了一整套动物行为学实验装置和图像处理软件系统,实现了大鼠肢体运动的捕捉和分析,重点对运动捕捉和目标跟踪方法进行了探索。
根据大鼠前肢骨架模型提出基于色块标记的实验方法以及相应的图像处理分析算法。本文研究了视觉系统的图像获取与转换、彩色图像分割、图像识别和目标跟踪等方面,方法中引入了Mean-shift算法,针对Mean-shift算法需要人工初始化和跟踪丢失等问题,加入目标检测和跟踪丢失处理机制,提出了一种基于改进Mean-shift算法的多目标跟踪算法,实现了大鼠肢体运动捕捉,提取关节角矢量作为肢体运动学特征参数,应用于运动分析。
以大鼠步态行为研究为例,对此方法进行了初步测试。实验结果表明本系统可有效描述其步态周期肢体运动轨迹,并将大鼠步态周期识别为三个基本运动单元：后摆离地期、前摆着地期和着地缓冲期。
Behavior analysis of the laboratory animal is a necessary part of the research of high-level nerve centers function. It is not only widely applied in neuropharmacology and neuropsychology but also have extended to the interdisciplinary fields, such as the bionics, artificial intelligent and brain computer interface. The parameterized description of the locomotor behavior is the basis of the behavior analysis, so to set up the method of automatic analysis of the animal behavior is significant for related research.
We have developed an animal behavioristics experimental facility and real-time image process software, according to the requirement of the characteristic of the rat's behavior and the forelimb moving behavior analysis. The Color-marker method is used for motion capture.
Software system consists of image acquiring, color image preprocessing, color image dividing, object identifying and tracking. First, the fast image segmentation algorithm is used to detect the marker from image. Then, markers are detected and tracked using improved Mean-shift algorithm. Finally, the angles of forelimb joint are used to describe the motion of forelimb.
We have used the automatic analysis method mentioned above to analyze rat's gait. The experiment results show that the software can accurately record forelimb motion trace. According to the angles of forelimb joint, rat's gait can be classified to three phases: Backswing-Heel-Off, forward swing, Initial Contact-ground.
实验大鼠的肢体运动具有目标小,运动变化快的特点,对肢体运动检测与分析带来了极大挑战。本文从精确性、稳定性和实时性出发,对实验装置设计、实验方法以及相关图像处理分析算法进行了研究。设计了一整套动物行为学实验装置和图像处理软件系统,实现了大鼠肢体运动的捕捉和分析,重点对运动捕捉和目标跟踪方法进行了探索。
根据大鼠前肢骨架模型提出基于色块标记的实验方法以及相应的图像处理分析算法。本文研究了视觉系统的图像获取与转换、彩色图像分割、图像识别和目标跟踪等方面,方法中引入了Mean-shift算法,针对Mean-shift算法需要人工初始化和跟踪丢失等问题,加入目标检测和跟踪丢失处理机制,提出了一种基于改进Mean-shift算法的多目标跟踪算法,实现了大鼠肢体运动捕捉,提取关节角矢量作为肢体运动学特征参数,应用于运动分析。
以大鼠步态行为研究为例,对此方法进行了初步测试。实验结果表明本系统可有效描述其步态周期肢体运动轨迹,并将大鼠步态周期识别为三个基本运动单元：后摆离地期、前摆着地期和着地缓冲期。
Behavior analysis of the laboratory animal is a necessary part of the research of high-level nerve centers function. It is not only widely applied in neuropharmacology and neuropsychology but also have extended to the interdisciplinary fields, such as the bionics, artificial intelligent and brain computer interface. The parameterized description of the locomotor behavior is the basis of the behavior analysis, so to set up the method of automatic analysis of the animal behavior is significant for related research.
We have developed an animal behavioristics experimental facility and real-time image process software, according to the requirement of the characteristic of the rat's behavior and the forelimb moving behavior analysis. The Color-marker method is used for motion capture.
Software system consists of image acquiring, color image preprocessing, color image dividing, object identifying and tracking. First, the fast image segmentation algorithm is used to detect the marker from image. Then, markers are detected and tracked using improved Mean-shift algorithm. Finally, the angles of forelimb joint are used to describe the motion of forelimb.
We have used the automatic analysis method mentioned above to analyze rat's gait. The experiment results show that the software can accurately record forelimb motion trace. According to the angles of forelimb joint, rat's gait can be classified to three phases: Backswing-Heel-Off, forward swing, Initial Contact-ground.
引文
[1]Herbin M, Hackert R, Gasc J.P, Renous S, Gait parameters of treadmill versus overground locomotion in mouse[J].Behav Brain Res,2007.181:173-176.
[2]昊宝,卓豫.动物行为检测技术进展及展望[J].中国行为医学科学,2001,10(3)：278-280
[3]李静,郭国祯,郭鹞,基于机器视觉的动物行为测量分析的自动化系统[J].中国医学理论与实践,2005,7(15)：1015-1017
[4]张敏,张恒义,郑筱祥,基于轮廓曲率和谱系聚类的大鼠体态自动识别[J].浙江大学学报工学版,2006,40(3)
[5]卓豫,吴宝明,王禾,动物行为监测系统在创伤应激障碍模型中的应用[J].医疗卫生装备,2006,27(2)：24-26
[6]DONOGH UE J P. Connecting cortex to machines:recent advances in brain interfaces [J]. Nature Neuroscience,2002,5(Suppl.):1085 1088.
[7]Pedro, Vitor, Luis, A comparison of two-dimensional and three-dimensional techniques for the determination of hindlimb kinematics during treadmill locomotion in rats following spinal cord injury[J]. Journal of Neuroscience Methods,2008,173:193-200
[8]李豪杰,林守勋,张勇东,基于视频的人体运动捕捉综述[J]；计算机辅助设计与图形学学报,2006,11
[9]罗忠祥,庄越挺,潘云鹤,基于视频的运动捕获[J];中国图像图形学报：A版,2002,7(8)：752-758
[10]Paulus,Dulawa,Ralph R.J, Behavioral organization is independent of locomotor activity in 129 and C57 mouse strains[J].Brain Research,1999,835:27-36.
[11]郑秀娟,大鼠运动行为的自动分析[D].浙江：浙江大学生物医学工程,2006
[12]David C. Post, Gait Analysis Review. [EB/OL]. http://www.nd.edu/～dpost/IntSyst/report1.pdf
[13]卓豫,吴宝明,基于运动和声音同步记录的动物行为活动监测系统[J]；中国行为医学科学,2001,10卷1期
[14]胡雪艳,恽晓平,步态分析在临床中的应用[J];中国康复理论与实践,2003,9(11)：677～679
[15]Jeroen R. Dijkstra, Marcel F. Meek, Peter H. Robinson, Albert Gramsbergen. Methods to evaluate functional nerve recovery in adult rats:walking track analysis, video analysis and the withdrawal reflex [J].Journal of Neuroscience Methods,96(2000),89-96
[16]Artur S.P,Vareja,Cabrita,Motion of the foot and ankle during the stance phase in rats[J].Muscle& Nerve,2002,11
[17]张韶岷,陈卫东,孙超,动物脑电-行为同步记录及分析系统[J].浙江大学学报工学版,2009,43(11)
[18]James Bruce,Tucker Balch,Manuela Veloso. Fast and Inexpensive Color Image Segmentation for Interactive Robots[A]. International Conference on intelligent Robots and Systems 2000.
[19]John K.Chapin, Karen A.Moxon. Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex[J]. nature neuroscience 7(1999) volume 2 no 7
[20]Christian Ethier,Laurent Brizzi. Linear Summation of Cat Motor Cortex Outputs[J]. The Journal of Neuroscience.2006 26(20):5574-5581
[21]Trevor Drew, Jacques-Etienne Andujar. Cortical mechanisms involved in visuomotor coordination during precision walking[J].Brain Research reviews 57(2008) 199-211
[22]费峥峰,赵雁南,一种快速彩色图像颜色分割算法[J].机器人技术,2007,23：10-2
[23]刘建良.RoboCup小型足球机器人视觉系统的构建[D]杭州：浙江大学控制理论与控制工程,2007
[24]郭忠武,王广志,刘永斌,丁辉,刘利坚,丁海曙,基于数字视频和数字图像处理的步态分析系统[J].航天医学与医学工程,2002,4
[25]胡雪艳,恽晓平,步态分析在临床中的应用[J].中国康复理论与实践,2003,9(11)：677～679
[26]卢昕.基于视频的人体运动分析[D].杭州：浙江大学生物医学工程,2003：17
[27]张敏.基于体态识别的动物行为自动分析研究与应用[D].杭州：浙江大学生物医学工程,2005
[28]Thomas B. Moeslund,Erik Granum. A Survey of Computer Vision-Based Human Motion Capture[J]. Computer Vision and Image Understanding 81,231-268 (2001)
[29]Rafael C. G., Richard E. W., Digital image processing[M]. Beijing:Electronic Industry Press,2007:34-75.
[30]Gary Bradski, Adrian Kaebler, Learning OpenCV[M].Nanjing:Southeast University Press,2009
[31]George Shepherd, David Kruglinski.Visual C++.NET技术内幕(第6版)[M].北京：清华大学出版社,2004
[32]钱惠敏,茅耀斌,王执铨.自动选择跟踪窗尺度的Mean-shift算法[J].中国图像图形学报,2007,12(2)：245-249.
[33]E.K.Blum,C.Haun,J.E.Ryan. A musculo-skeletal model of rat ankle motion and its experimental test on rat[J]. Journal of Biomechanics,2007,40; 891-899
[34]肖栋.基于视频图像的运动目标检测与跟踪技术研究[D].武汉：华中科技大学控制理论与控制工程,2007
[35]刘国翌.基于视频的人体运动跟踪技术研究[D].北京：中国科学院计算技术研究所,2005
[36]达选福,张伯珩,边川平.高速CCD图像数据存储技术[J].光子学报,2003,11(32)：1393-1395
[37]吴晓阳.基于OpenCV的运动目标检测与跟踪[D].杭州：浙江大学电路与系统,2008
[38]贾静平.图像序列中多目标跟踪技术研究[D].西安：西北工业大学计算机应用技术,2004
[39]吴向平.视觉跟踪算法的研究及其在生物运动分析中的应用[D].杭州：浙江大学生物医学工程,2004
[40]郭康德.基于视觉的三维指尖检测算法和应用[D].杭州：浙江大学计算机应用技术,2010
[41]Ning Song Peng,Jie Yang,Zhi Liu. Mean shift blob tracking with kernel histogram filtering and hypothesis testing[J].Pattern Recognition Letters,2005,26:605-614
[2]昊宝,卓豫.动物行为检测技术进展及展望[J].中国行为医学科学,2001,10(3)：278-280
[3]李静,郭国祯,郭鹞,基于机器视觉的动物行为测量分析的自动化系统[J].中国医学理论与实践,2005,7(15)：1015-1017
[4]张敏,张恒义,郑筱祥,基于轮廓曲率和谱系聚类的大鼠体态自动识别[J].浙江大学学报工学版,2006,40(3)
[5]卓豫,吴宝明,王禾,动物行为监测系统在创伤应激障碍模型中的应用[J].医疗卫生装备,2006,27(2)：24-26
[6]DONOGH UE J P. Connecting cortex to machines:recent advances in brain interfaces [J]. Nature Neuroscience,2002,5(Suppl.):1085 1088.
[7]Pedro, Vitor, Luis, A comparison of two-dimensional and three-dimensional techniques for the determination of hindlimb kinematics during treadmill locomotion in rats following spinal cord injury[J]. Journal of Neuroscience Methods,2008,173:193-200
[8]李豪杰,林守勋,张勇东,基于视频的人体运动捕捉综述[J]；计算机辅助设计与图形学学报,2006,11
[9]罗忠祥,庄越挺,潘云鹤,基于视频的运动捕获[J];中国图像图形学报：A版,2002,7(8)：752-758
[10]Paulus,Dulawa,Ralph R.J, Behavioral organization is independent of locomotor activity in 129 and C57 mouse strains[J].Brain Research,1999,835:27-36.
[11]郑秀娟,大鼠运动行为的自动分析[D].浙江：浙江大学生物医学工程,2006
[12]David C. Post, Gait Analysis Review. [EB/OL]. http://www.nd.edu/～dpost/IntSyst/report1.pdf
[13]卓豫,吴宝明,基于运动和声音同步记录的动物行为活动监测系统[J]；中国行为医学科学,2001,10卷1期
[14]胡雪艳,恽晓平,步态分析在临床中的应用[J];中国康复理论与实践,2003,9(11)：677～679
[15]Jeroen R. Dijkstra, Marcel F. Meek, Peter H. Robinson, Albert Gramsbergen. Methods to evaluate functional nerve recovery in adult rats:walking track analysis, video analysis and the withdrawal reflex [J].Journal of Neuroscience Methods,96(2000),89-96
[16]Artur S.P,Vareja,Cabrita,Motion of the foot and ankle during the stance phase in rats[J].Muscle& Nerve,2002,11
[17]张韶岷,陈卫东,孙超,动物脑电-行为同步记录及分析系统[J].浙江大学学报工学版,2009,43(11)
[18]James Bruce,Tucker Balch,Manuela Veloso. Fast and Inexpensive Color Image Segmentation for Interactive Robots[A]. International Conference on intelligent Robots and Systems 2000.
[19]John K.Chapin, Karen A.Moxon. Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex[J]. nature neuroscience 7(1999) volume 2 no 7
[20]Christian Ethier,Laurent Brizzi. Linear Summation of Cat Motor Cortex Outputs[J]. The Journal of Neuroscience.2006 26(20):5574-5581
[21]Trevor Drew, Jacques-Etienne Andujar. Cortical mechanisms involved in visuomotor coordination during precision walking[J].Brain Research reviews 57(2008) 199-211
[22]费峥峰,赵雁南,一种快速彩色图像颜色分割算法[J].机器人技术,2007,23：10-2
[23]刘建良.RoboCup小型足球机器人视觉系统的构建[D]杭州：浙江大学控制理论与控制工程,2007
[24]郭忠武,王广志,刘永斌,丁辉,刘利坚,丁海曙,基于数字视频和数字图像处理的步态分析系统[J].航天医学与医学工程,2002,4
[25]胡雪艳,恽晓平,步态分析在临床中的应用[J].中国康复理论与实践,2003,9(11)：677～679
[26]卢昕.基于视频的人体运动分析[D].杭州：浙江大学生物医学工程,2003：17
[27]张敏.基于体态识别的动物行为自动分析研究与应用[D].杭州：浙江大学生物医学工程,2005
[28]Thomas B. Moeslund,Erik Granum. A Survey of Computer Vision-Based Human Motion Capture[J]. Computer Vision and Image Understanding 81,231-268 (2001)
[29]Rafael C. G., Richard E. W., Digital image processing[M]. Beijing:Electronic Industry Press,2007:34-75.
[30]Gary Bradski, Adrian Kaebler, Learning OpenCV[M].Nanjing:Southeast University Press,2009
[31]George Shepherd, David Kruglinski.Visual C++.NET技术内幕(第6版)[M].北京：清华大学出版社,2004
[32]钱惠敏,茅耀斌,王执铨.自动选择跟踪窗尺度的Mean-shift算法[J].中国图像图形学报,2007,12(2)：245-249.
[33]E.K.Blum,C.Haun,J.E.Ryan. A musculo-skeletal model of rat ankle motion and its experimental test on rat[J]. Journal of Biomechanics,2007,40; 891-899
[34]肖栋.基于视频图像的运动目标检测与跟踪技术研究[D].武汉：华中科技大学控制理论与控制工程,2007
[35]刘国翌.基于视频的人体运动跟踪技术研究[D].北京：中国科学院计算技术研究所,2005
[36]达选福,张伯珩,边川平.高速CCD图像数据存储技术[J].光子学报,2003,11(32)：1393-1395
[37]吴晓阳.基于OpenCV的运动目标检测与跟踪[D].杭州：浙江大学电路与系统,2008
[38]贾静平.图像序列中多目标跟踪技术研究[D].西安：西北工业大学计算机应用技术,2004
[39]吴向平.视觉跟踪算法的研究及其在生物运动分析中的应用[D].杭州：浙江大学生物医学工程,2004
[40]郭康德.基于视觉的三维指尖检测算法和应用[D].杭州：浙江大学计算机应用技术,2010
[41]Ning Song Peng,Jie Yang,Zhi Liu. Mean shift blob tracking with kernel histogram filtering and hypothesis testing[J].Pattern Recognition Letters,2005,26:605-614