基于CNN的相衬显微图像序列的癌细胞多目标跟踪
|
摘要
检测与跟踪相衬显微图像序列下的癌细胞对于分析癌细胞的生命周期以及开发抗癌新药具有非常重要的意义。传统的目标跟踪方法大多应用于刚性目标跟踪或单目标跟踪,而癌细胞是非刚性且不断裂变的多目标,这就大大增加了跟踪的难度。文中以相衬显微图像序列中的膀胱癌细胞为研究对象,提出一种基于卷积神经网络(Convolutional Neural Network,CNN)的癌细胞多目标跟踪方法。该算法采用基于检测的多目标跟踪方法,首先利用深度学习检测框架Faster R-CNN卷积神经网络实现癌细胞的检测,初步获得待跟踪的癌细胞;再利用扫描圆算法(Circle Scanning Algorithm,CSA)实现黏连细胞的检测优化,进一步提高黏连区域的细胞检测精度;最后提取综合特征描述子,对卷积特征、尺寸特征和位置特征进行加权求和,实现跟踪目标的综合描述,从而实现不同帧癌细胞间的高效关联匹配,最终实现癌细胞的多目标跟踪。一系列实验结果表明,相较于传统方法,所提方法不仅在癌细胞的检测和跟踪上性能有较大的提升,而且可以有效处理目标的遮挡问题。
Detecting and tracking cancer cells under phase contrast microscopic images plays a critical role for analyzing the life cycle of cancer cells and developing new anti-cancer drugs.Traditional target tracking methods are mostly applied to rigid target tracking or single target tracking,while cancer cells are non-rigid multiple targets with constant fission,and it makes tracking more challenging.Taking bladder cancer cells in the sequence of phase contrast micrographs images as research object,this paper proposed a multi-target tracking method of cancer cells based on convolutional neural network.Firstly,through making use of detection-based multi-target method,the proposed algorithm adopted the deep learning detection framework Faster R-CNN to detect the bladder cancer cells and preliminarily obtain the cancer cells to be tracked.Then CSA(circle scanning algorithm) was utilized to further optimize the detection of adhesion cancer cells,and further improve the detection accuracy of cells in adhesion area.Finally,it integrated the features of convolution,size and position into a synthetic feature descriptor by using weighting methods,thus tracking multiple cancer cells by achieving the efficient correlation and matching of different frames of cancer cells.The results of a series of experiments show that this method can not only improve the accuracy of detecting and tracking cancer cell,but also deal with the occlusion problem effectively.
Detecting and tracking cancer cells under phase contrast microscopic images plays a critical role for analyzing the life cycle of cancer cells and developing new anti-cancer drugs.Traditional target tracking methods are mostly applied to rigid target tracking or single target tracking,while cancer cells are non-rigid multiple targets with constant fission,and it makes tracking more challenging.Taking bladder cancer cells in the sequence of phase contrast micrographs images as research object,this paper proposed a multi-target tracking method of cancer cells based on convolutional neural network.Firstly,through making use of detection-based multi-target method,the proposed algorithm adopted the deep learning detection framework Faster R-CNN to detect the bladder cancer cells and preliminarily obtain the cancer cells to be tracked.Then CSA(circle scanning algorithm) was utilized to further optimize the detection of adhesion cancer cells,and further improve the detection accuracy of cells in adhesion area.Finally,it integrated the features of convolution,size and position into a synthetic feature descriptor by using weighting methods,thus tracking multiple cancer cells by achieving the efficient correlation and matching of different frames of cancer cells.The results of a series of experiments show that this method can not only improve the accuracy of detecting and tracking cancer cell,but also deal with the occlusion problem effectively.
引文
[1] M?LLER M,BURGER M,DIETERICH P,et al.A framework for automated cell tracking in phase contrast microscopic videos based on normal velocities[J].Journal of Visual Communication &Image Representation,2012,25(2):396-409.
[2] LI H,LI Y,PORIKLI F.Robust Online Visual Tracking with a Single Convolutional Neural Network [M]//Computer Vision -- ACCV 2014.Springer International Publishing,2014:194-209.
[3] WANG N,YEUNG D Y.Learning a deep compact image representation for visual tracking [C]//International Conference on Neural Information Processing Systems.2013:809-817.
[4] HU H G,LUO C,GUAN Q,et al.A fast online multivariable identification method for greenhouse environment control problems [J].Neurocomputing,2018,312:63-73.
[5] GIRSHICK R,DONAHUE J,DARRELL T,et al.Region-Based Convolutional Networks for Accurate Object Detection and Segmentation [J].IEEE Trans Pattern Anal Mach Intell,2016,38(1):142-158.
[6] OPELT A,FUSSENEGGER M,PINZ A,et al.Weak Hypotheses and Boosting for Generic Object Detection and Recognition[C]//European Conference on Computer Vision.Springer Berlin Heidelberg,2004:71-84.
[7] DING S,LIU Z,LI C.AdaBoost learning for fabric defect detection based on HOG and SVM[C]//International Conference on Multimedia Technology.IEEE,2011:2903-2906.
[8] KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks [J].Advances in Neural Information Processing Systems,2012,25(2):1097-1105.
[9] REN S,HE K,GIRSHICK R,et al.Faster R-CNN:Towards Real-Time Object Detection with Region Proposal Networks [J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2015,1:91-99.
[10] GIRSHICK R,DONAHUE J,DARRELL T,et al.Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation[C]//IEEE Conference on Computer Vision and Pattern Recognition.2014:580-587.
[11] GIRSHICK R.Fast r-cnn[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:1440-1448.
[12] REDMON J,DIVVALA S,GIRSHICK R,et al.You Only Look Once:Unified,Real-Time Object Detection[J].Computer Scie-nce,2016:779-788.
[13] LIU W,ANGUELOV D,ERHAN D,et al.SSD:Single Shot MultiBox Detector[C]//European Conference on Computer Vision.Springer International Publishing,2016:21-37.
[14] UIJLINGS J R R,SANDE K E A V D,GEVERS T,et al.Selective Search for Object Recognition [J].International Journal of Computer Vision,2013,104(2):154-171.
[15] HU H G,GUAN Q,CHEN S Y,et al.Detection and Recognition for Life State of Cell Cancer Using Two-Stage Cascade CNNs [J].IEEE/ACM Transactions on Computational Biology and Bioinformatics,2017:1.
[16] CHU H,WANG K.Target tracking based on mean shift and improved kalman filtering algorithm[C]//IEEE International Conference on Automation and Logistics.IEEE,2009:808-812.
[17] MA L,CHANG F L,QIAO Y Z.Target tracking based on mean shift algorithm and particle filtering algorithm [J].Pattern Re-cognition & Artificial Intelligence,2006,19(6):787-793.
[18] KALAL Z,MIKOLAJCZYK K,MATAS J.Tracking-learning-detection [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(7):1409-1422.
[19] JIANG L H,GAN Z H,JIANG M.Review of Multi-Target Tracking[J].Computer Systems & Applications,2010,19(12):271-275.
[20] LAURITSCH G,DENNERLEIN F.Method for reconstructing a CT image using an algorithm for a short-scan circle combined with various lines:US,US 7359477 B2[P].2008.
[21] TAKALA V,PIETIKAINEN M.Multi-Object Tracking Using Color,Texture and Motion[C]//IEEE Computer Society Conference on Computer Vision and Pattern Recognition.DBLP,2007:1-7.
[22] WANG L,WANG J,CHEN D,et al.An Algorithm of Target Tracking and Detecting Synchronously Based on Inter-Frame Dual Match [C]//2009 Symposium on Photonics and Optoelectronics,2009:1-4.
[23] CHEN L,ZHAO Z,YAN H.A Probabilistic Relaxation Labeling (PRL) Based Method for C.elegans Cell Tracking in Microscopic Image Sequences [J].IEEE Journal of Selected Topics in Signal Processing,2016,10(1):185-192.
[24] CHOI W.Near-online multi-target tracking with aggregated local flow descriptor[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:3029-3037.
[25] MEMISEVIC R,ZACH C,HINTON G,et al.Gated Softmax classification[C]//International Conference on Neural Information Processing Systems.Curran Associates Inc.2010:1603-1611.
[26] TAN Y.Study on applied technology arithmetic of image threshold segmentation [J].Microcomputer Information,2007,23(24):298-300.
[27] SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition [J].Computer Scien-ce,arXiv:1409.1556.
[28] ZEILER M D,FERGUS R.Visualizing and understanding con- volutional networks[C]//European Conference on Computer Vision.Springer International Publishing,2014:818-833.
[29] LARKIN J,GUNTHER B,HOLIERHOEK H.NMS with multi-server change requests processing:US,US 8028052 B2[P].2011.
[30] WU Y,LIM J,YANG M H.Object Tracking Benchmark [J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2015,37(9):1834-1848.
[31] BERNARDIN K,STIEFELHAGEN R.Evaluating multiple object tracking performance:the CLEAR MOT metrics [J].Eu-rasip Journal on Image & Video Processing,2008,2008(1):246309.
[32] 张剑华,刘儒瑜,邹祎杰.显微图像序列中癌细胞轨迹追踪与关联[OL].http://www.paper.edu.cn/releasepaper/content/201704-582.
[33] JU H Y,YANG M H,LIM J,et al.Bayesian Multi-object Tracking Using Motion Context from Multiple Objects[C]//IEEE Winter Conference on Applications of Computer Vision.IEEE Computer Society,2015:33-40.
[34] FAZLI S,POUR H M,BOUZARI H.Multiple object tracking using improved GMM-based motion segmentation[C]//International Conference on Electrical Engineering/Electronics,Computer,Telecommunications and Information Technology,Ecti-Con.IEEE,2009:1130-1133.
[2] LI H,LI Y,PORIKLI F.Robust Online Visual Tracking with a Single Convolutional Neural Network [M]//Computer Vision -- ACCV 2014.Springer International Publishing,2014:194-209.
[3] WANG N,YEUNG D Y.Learning a deep compact image representation for visual tracking [C]//International Conference on Neural Information Processing Systems.2013:809-817.
[4] HU H G,LUO C,GUAN Q,et al.A fast online multivariable identification method for greenhouse environment control problems [J].Neurocomputing,2018,312:63-73.
[5] GIRSHICK R,DONAHUE J,DARRELL T,et al.Region-Based Convolutional Networks for Accurate Object Detection and Segmentation [J].IEEE Trans Pattern Anal Mach Intell,2016,38(1):142-158.
[6] OPELT A,FUSSENEGGER M,PINZ A,et al.Weak Hypotheses and Boosting for Generic Object Detection and Recognition[C]//European Conference on Computer Vision.Springer Berlin Heidelberg,2004:71-84.
[7] DING S,LIU Z,LI C.AdaBoost learning for fabric defect detection based on HOG and SVM[C]//International Conference on Multimedia Technology.IEEE,2011:2903-2906.
[8] KRIZHEVSKY A,SUTSKEVER I,HINTON G E.ImageNet classification with deep convolutional neural networks [J].Advances in Neural Information Processing Systems,2012,25(2):1097-1105.
[9] REN S,HE K,GIRSHICK R,et al.Faster R-CNN:Towards Real-Time Object Detection with Region Proposal Networks [J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2015,1:91-99.
[10] GIRSHICK R,DONAHUE J,DARRELL T,et al.Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation[C]//IEEE Conference on Computer Vision and Pattern Recognition.2014:580-587.
[11] GIRSHICK R.Fast r-cnn[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:1440-1448.
[12] REDMON J,DIVVALA S,GIRSHICK R,et al.You Only Look Once:Unified,Real-Time Object Detection[J].Computer Scie-nce,2016:779-788.
[13] LIU W,ANGUELOV D,ERHAN D,et al.SSD:Single Shot MultiBox Detector[C]//European Conference on Computer Vision.Springer International Publishing,2016:21-37.
[14] UIJLINGS J R R,SANDE K E A V D,GEVERS T,et al.Selective Search for Object Recognition [J].International Journal of Computer Vision,2013,104(2):154-171.
[15] HU H G,GUAN Q,CHEN S Y,et al.Detection and Recognition for Life State of Cell Cancer Using Two-Stage Cascade CNNs [J].IEEE/ACM Transactions on Computational Biology and Bioinformatics,2017:1.
[16] CHU H,WANG K.Target tracking based on mean shift and improved kalman filtering algorithm[C]//IEEE International Conference on Automation and Logistics.IEEE,2009:808-812.
[17] MA L,CHANG F L,QIAO Y Z.Target tracking based on mean shift algorithm and particle filtering algorithm [J].Pattern Re-cognition & Artificial Intelligence,2006,19(6):787-793.
[18] KALAL Z,MIKOLAJCZYK K,MATAS J.Tracking-learning-detection [J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(7):1409-1422.
[19] JIANG L H,GAN Z H,JIANG M.Review of Multi-Target Tracking[J].Computer Systems & Applications,2010,19(12):271-275.
[20] LAURITSCH G,DENNERLEIN F.Method for reconstructing a CT image using an algorithm for a short-scan circle combined with various lines:US,US 7359477 B2[P].2008.
[21] TAKALA V,PIETIKAINEN M.Multi-Object Tracking Using Color,Texture and Motion[C]//IEEE Computer Society Conference on Computer Vision and Pattern Recognition.DBLP,2007:1-7.
[22] WANG L,WANG J,CHEN D,et al.An Algorithm of Target Tracking and Detecting Synchronously Based on Inter-Frame Dual Match [C]//2009 Symposium on Photonics and Optoelectronics,2009:1-4.
[23] CHEN L,ZHAO Z,YAN H.A Probabilistic Relaxation Labeling (PRL) Based Method for C.elegans Cell Tracking in Microscopic Image Sequences [J].IEEE Journal of Selected Topics in Signal Processing,2016,10(1):185-192.
[24] CHOI W.Near-online multi-target tracking with aggregated local flow descriptor[C]//Proceedings of the IEEE International Conference on Computer Vision.2015:3029-3037.
[25] MEMISEVIC R,ZACH C,HINTON G,et al.Gated Softmax classification[C]//International Conference on Neural Information Processing Systems.Curran Associates Inc.2010:1603-1611.
[26] TAN Y.Study on applied technology arithmetic of image threshold segmentation [J].Microcomputer Information,2007,23(24):298-300.
[27] SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large-Scale Image Recognition [J].Computer Scien-ce,arXiv:1409.1556.
[28] ZEILER M D,FERGUS R.Visualizing and understanding con- volutional networks[C]//European Conference on Computer Vision.Springer International Publishing,2014:818-833.
[29] LARKIN J,GUNTHER B,HOLIERHOEK H.NMS with multi-server change requests processing:US,US 8028052 B2[P].2011.
[30] WU Y,LIM J,YANG M H.Object Tracking Benchmark [J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2015,37(9):1834-1848.
[31] BERNARDIN K,STIEFELHAGEN R.Evaluating multiple object tracking performance:the CLEAR MOT metrics [J].Eu-rasip Journal on Image & Video Processing,2008,2008(1):246309.
[32] 张剑华,刘儒瑜,邹祎杰.显微图像序列中癌细胞轨迹追踪与关联[OL].http://www.paper.edu.cn/releasepaper/content/201704-582.
[33] JU H Y,YANG M H,LIM J,et al.Bayesian Multi-object Tracking Using Motion Context from Multiple Objects[C]//IEEE Winter Conference on Applications of Computer Vision.IEEE Computer Society,2015:33-40.
[34] FAZLI S,POUR H M,BOUZARI H.Multiple object tracking using improved GMM-based motion segmentation[C]//International Conference on Electrical Engineering/Electronics,Computer,Telecommunications and Information Technology,Ecti-Con.IEEE,2009:1130-1133.