基于卷积神经网络的冬小麦麦穗检测计数系统
|
摘要
为进一步提高大田环境下麦穗识别与检测计数的准确性,基于图像处理和深度学习技术,设计并实现了基于卷积神经网络的冬小麦麦穗检测计数系统。根据大田环境下采集的开花期冬小麦图像特点,提取麦穗、叶片、阴影3类标签图像构建数据集,研究适用于冬小麦麦穗识别的卷积神经网络结构,构建了冬小麦麦穗识别模型,并采用梯度下降法对模型进行训练;将构建的冬小麦麦穗识别模型与非极大值抑制结合,进行冬小麦麦穗计数。试验结果表明,该系统构建的冬小麦麦穗识别模型能够有效地克服大田环境下的噪声,实现麦穗的快速、准确识别,总体识别正确率达到99. 6%,其中麦穗识别正确率为99. 9%,阴影识别正确率为99. 7%,叶片识别正确率为99. 3%。对100幅冬小麦图像进行麦穗计数测试,采用决定系数和归一化均方根误差(NRMSE)进行正确率定量评价,结果表明,该系统计数结果与人工计数结果线性拟合的R~2为0. 62,NRMSE为11. 73%,能够满足冬小麦麦穗检测计数的实际要求。
The ear of winter wheat,as an important agronomic component,is not only closely associated with yield,but also plays an important role in phenotypic analysis. It was reported that the number of winter wheat ears per unit area was one of the commonly used indicators to indicate the winter wheat yield. However,the traditional manual counting method is time-consuming and labor-intensive,as well as subjective,lacking a unified winter wheat ear counting standard. In order to increase the accuracy of winter wheat ear recognition and detection in field condition,a winter wheat ear detection system was constructed based on image processing and deep learning. Firstly,a winter wheat ear recognition model was proposed, which was based on manual image segmentation and convolutional neural network classification. A 27-layer network with five convolutional layers,four pooling layers and two fully connected layers was constructed. The gradient descending method( SGD) was used to train and validate the model by setting the maximum number of epochs at 200. The network was trained with an initial learning rate of 0. 001. In the winter wheat ear detection and counting stage,a non-maximal suppression( NMS) method was used to overcome the effect of overlapping results by using a confidence score. The confidence score p was set to be 0. 95,and the I threshold was set to be 0. 1. The results showed that the system achieved an overall recognition accuracy of 99. 6%,99. 9% for winter wheat ear,99. 7% for shadow and 99. 3% for leaf,which indicated that the winter wheat ear detection system was capable of recognizing winter wheat ears. The linear regression was used to test the accuracy of the counting results.Normalized root mean squared error( NRMSE) and coefficient of determination( R~2) were used as the criterion for evaluation. The comparison between the counting results by the system of the selected 100 photos and the manual counting results showed that R~2 was 0. 62 and NRMSE was 11. 73%. It was revealed that the accuracy of winter wheat ears could be achieved by the system,which can provide support to yield estimation and field management of winter wheat.
The ear of winter wheat,as an important agronomic component,is not only closely associated with yield,but also plays an important role in phenotypic analysis. It was reported that the number of winter wheat ears per unit area was one of the commonly used indicators to indicate the winter wheat yield. However,the traditional manual counting method is time-consuming and labor-intensive,as well as subjective,lacking a unified winter wheat ear counting standard. In order to increase the accuracy of winter wheat ear recognition and detection in field condition,a winter wheat ear detection system was constructed based on image processing and deep learning. Firstly,a winter wheat ear recognition model was proposed, which was based on manual image segmentation and convolutional neural network classification. A 27-layer network with five convolutional layers,four pooling layers and two fully connected layers was constructed. The gradient descending method( SGD) was used to train and validate the model by setting the maximum number of epochs at 200. The network was trained with an initial learning rate of 0. 001. In the winter wheat ear detection and counting stage,a non-maximal suppression( NMS) method was used to overcome the effect of overlapping results by using a confidence score. The confidence score p was set to be 0. 95,and the I threshold was set to be 0. 1. The results showed that the system achieved an overall recognition accuracy of 99. 6%,99. 9% for winter wheat ear,99. 7% for shadow and 99. 3% for leaf,which indicated that the winter wheat ear detection system was capable of recognizing winter wheat ears. The linear regression was used to test the accuracy of the counting results.Normalized root mean squared error( NRMSE) and coefficient of determination( R~2) were used as the criterion for evaluation. The comparison between the counting results by the system of the selected 100 photos and the manual counting results showed that R~2 was 0. 62 and NRMSE was 11. 73%. It was revealed that the accuracy of winter wheat ears could be achieved by the system,which can provide support to yield estimation and field management of winter wheat.
引文
[1]龚红菊,姬长英.基于图像处理技术的麦穗产量测量方法[J].农业机械学报,2007,38(12):116-119.GONG Hongju,JI Changying. Wheat ear yield measurement method based on image processing technology[J]. Transactions of the Chinese Society for Agricultural Machinery,2007,38(12):116-119.(in Chinese)
[2]陈含,吕行军,田凤珍,等.基于Sobel算子边缘检测的麦穗图像分割[J].农机化研究,2013,35(3):33-36.CHEN Han,LXingjun,TIAN Fengzhen,et al. Wheat image segmentation based on Sobel operator edge detection[J].Agricultural Mechanization Research,2013,35(3):33-36.(in Chinese)
[3] FERNANDEZ-GALLEGO J A,KEFAUVER S C,GUTIERREZ N A,et al. Wheat ear counting in-field conditions:high throughput and low-cost approach using RGB images[J]. Plant Methods,2018,14:22-33.
[4]赵锋,王克俭,苑迎春.基于颜色特征和Ada Boost算法的麦穗识别的研究[J].作物杂志,2014(1):141-144.ZHAO Feng,WANG Kejian,YUAN Yingchun. Research on wheat ear recognition based on color feature and Ada Boost algorithm[J]. Crops,2014(1):141-144.(in Chinese)
[5]刘涛,孙成明,王力坚,等.基于图像处理技术的大田麦穗计数[J/OL].农业机械学报,2014,45(2):282-290.LIU Tao,SUN Chengming,WANG Lijian,et al. Field wheat ear count based on image processing technology[J/OL].Transactions of the Chinese Society for Agricultural Machinery,2014,45(2):282-290. http:∥www. j-csam. org/jcsam/ch/reader/view_abstract. aspx? flag=1&file_no=20140247&journal_id=jcsam. DOI:10. 6041/j. issn. 1000-1298. 2014. 02. 047.(in Chinese)
[6]范梦扬,马钦,刘峻明,等.基于机器视觉的大田环境小麦麦穗计数方法[J/OL].农业机械学报,2015,46(增刊):234-239.FAN Mengyang,MA Qin,LIU Junming,et al. Wheat vision counting method for field environment based on machine vision[J/OL]. Transactions of the Chinese Society for Agricultural Machinery,2015,46(Supp.):234-239. http:∥www. j-csam. org/jcsam/ch/reader/view_abstract. aspx? flag=1&file_no=2015S038&journal_id=jcsam. DOI:10. 6041/j. issn. 1000-1298.2015. S0. 038.(in Chinese)
[7]周云成,许童羽,郑伟,等.基于深度卷积神经网络的番茄主要器官分类识别方法[J].农业工程学报,2017,33(15):219-226.ZHOU Yuncheng,XU Tongyu,ZHENG Wei,et al. Classification and recognition method of tomato main organs based on deep convolutional neural network[J]. Transactions of the CSAE,2017,33(15):219-226.(in Chinese)
[8]高震宇,王安,刘勇,等.基于卷积神经网络的鲜茶叶智能分选系统研究[J/OL].农业机械学报,2017,48(7):53-58.GAO Zhenyu,WANG An,LIU Yong,et al. Intelligent fresh-tea-leaves sorting system research based on convolution neural network[J/OL]. Transactions of the Chinese Society for Agricultural Machinery,2017,48(7):53-58. http:∥www. j-csam.org/jcsam/ch/reader/view_abstract. aspx? flag=1&file_no=20170707&journal_id=jcsam. DOI:10. 6041/j. issn. 1000-1298.2017. 07. 007.(in Chinese)
[9] LU Y,YI S,ZENG N,et al. Identification of rice diseases using deep convolutional neural networks[J]. Neurocomputing,2017,267:378-384.
[10] JOHANNES A,PICON A,ALVAREZ-GILA A. Automatic plant disease diagnosis using mobile capture devices,applied on a wheat use case[J]. Computers and Electronics in Agriculture,2017,138:200-209.
[11] LU H,CAO Z,XIAO Y,et al. Tassel Net:counting maize tassels in the wild via local counts regression network[J]. Plant Methods,2017,13:79-95.
[12] UBBENS J,CIESLAK M,PRUSINKIEWICZ P,et al. The use of plant models in deep learning:an application to leaf counting in rosette plants[J]. Plant Methods,2018,14:6-15.
[13] TANG J,WANG D,ZHANG Z,et al. Weed identification based on K-means feature learning combined with convolutional neural network[J]. Computers and Electronics in Agriculture,2017,135:63-70.
[14] FERREIRA A D S,FREITAS D M,SILVA G G D,et al. Weed detection in soybean crops using Conv Nets[J]. Computers and Electronics in Agriculture,2017,143:314-324.
[15] GHAZI M M,YANIKOGLU B,APTOULA E. Plant identification using deep neural networks via optimization of transfer learning parameters[J]. Neurocomputing,2017,235:228-235.
[16] DYRMANN M,KARSTOFT H,MIDTIBY H S. Plant species classification using deep convolutional neural network[J].Biosystems Engineering,2016,151:72-80.
[17] RUβWURM M,KORNER M. Multi-temporal land cover classification with long short-term memory neural networks[J].ISPRS—International Archives of the Photogrammetry,Remote Sensing and Spatial Information Sciences,2017,42:551-558.
[18] KAMILARIS A,PRENAFETA-BOLDúF X. Deep learning in agriculture:a survey[J]. Computers and Electronics in Agriculture,2018,147:70-90.
[19] XIONG X,DUAN L,LIU L,et al. Panicle-SEG:a robust image segmentation method for rice panicles in the field based on deep learning and superpixel optimization[J]. Plant Methods,2017,13:104-118.
[20]马浚诚,温皓杰,李鑫星,等.基于图像处理的温室黄瓜霜霉病诊断系统[J/OL].农业机械学报,2017,48(2):195-202.MA Juncheng,WEN Haojie,LI Xinxing,et al. Greenhouse cucumber downy mildew diagnosis system based on image processing[J/OL]. Transactions of the Chinese Society for Agricultural Machinery,2017,48(2):195-202. http:∥www. jcsam. org/jcsam/ch/reader/view_abstract. aspx? flag=1&file_no=20170226&journal_id=jcsam. DOI:10. 6041/j. issn.1000-1298. 2017. 02. 026.(in Chinese)
[21]马浚诚,李鑫星,温皓杰,等.面向叶类蔬菜病害识别的温室监控视频采集系统[J/OL].农业机械学报,2015,46(3):282-287.MA Juncheng,LI Xinxing,WEN Haojie,et al. Monitoring video capture system for identification of greenhouse vegetable diseases[J/OL]. Transactions of the Chinese Society for Agricultural Machinery,2015,46(3):282-287. http:∥www. jcsam. org/jcsam/ch/reader/view_abstract. aspx? flag=1&file_no=20150341&journal_id=jcsam. DOI:10. 6041/j. issn.1000-1298. 2015. 03. 041.(in Chinese)
[22]马浚诚,杜克明,郑飞翔,等.基于卷积神经网络的温室黄瓜病害识别系统[J].农业工程学报,2018,34(12):186-192.MA Juncheng, DU Keming, ZHENG Feixiang, et al. Greenhouse cucumber disease identification system based on convolutional neural network[J]. Transactions of the CSAE,2018,34(12):186-192.(in Chinese)
[23] LECUN Y,BENGIO Y,HINTON G. Deep learning[J]. Nature,2015,521(7553):436-444.
[24] LIU X,LIU W,LIU Y,et al. A survey of deep neural network architectures and their applications[J]. Neurocomputing,2017,234:11-26.
[25] GHOSAL S,BLYSTONE D,SINGH A K,et al. An explainable deep machine vision framework for plant stress phenotyping[J]. Proceedings of the National Academy of Sciences,2018,115(18):4613-4618.
[26] SRIVASTAVA N H G K A. Dropout:a simple way to prevent neural networks from overfitting[J]. Journal of Machine Learning Research,2014,15(1):1929-1958.
[27] HOSANG J,BENENSON R,SCHIELE B. Learning non-maximum suppression[C]∥IEEE Conference on Computer Vision&Pattern Recognition,2017:6469-6477.
[28] BODLA N,SINGH B,CHELLAPPA R,et al. Improving object detection with one line of code[C]∥Proceedings of the IEEE International Conference on Computer Vision,2017:5562-5570.
[29] VEDALDI A,LENC K. MatConvNet:convolutional neural networks for MATLAB[C]∥ACM International Conference on Multimedia,ACM,2015:689-692.
[30] MA J,DU K,ZHENG F,et al. A recognition method for cucumber diseases using leaf symptom images based on deep convolutional neural network[J]. Computers and Electronics in Agriculture,2018,154:18-24.
[31] DING W,TAYLOR G. Automatic moth detection from trap images for pest management[J]. Computers and Electronics in Agriculture,2016,123:17-28.
[32] MA J,LI Y,DU K,et al. Estimating above ground biomass of winter wheat at early growth stages using digital images and deep convolutional neural network[J]. European Journal of Agronomy,2019,103:117-129.
[2]陈含,吕行军,田凤珍,等.基于Sobel算子边缘检测的麦穗图像分割[J].农机化研究,2013,35(3):33-36.CHEN Han,LXingjun,TIAN Fengzhen,et al. Wheat image segmentation based on Sobel operator edge detection[J].Agricultural Mechanization Research,2013,35(3):33-36.(in Chinese)
[3] FERNANDEZ-GALLEGO J A,KEFAUVER S C,GUTIERREZ N A,et al. Wheat ear counting in-field conditions:high throughput and low-cost approach using RGB images[J]. Plant Methods,2018,14:22-33.
[4]赵锋,王克俭,苑迎春.基于颜色特征和Ada Boost算法的麦穗识别的研究[J].作物杂志,2014(1):141-144.ZHAO Feng,WANG Kejian,YUAN Yingchun. Research on wheat ear recognition based on color feature and Ada Boost algorithm[J]. Crops,2014(1):141-144.(in Chinese)
[5]刘涛,孙成明,王力坚,等.基于图像处理技术的大田麦穗计数[J/OL].农业机械学报,2014,45(2):282-290.LIU Tao,SUN Chengming,WANG Lijian,et al. Field wheat ear count based on image processing technology[J/OL].Transactions of the Chinese Society for Agricultural Machinery,2014,45(2):282-290. http:∥www. j-csam. org/jcsam/ch/reader/view_abstract. aspx? flag=1&file_no=20140247&journal_id=jcsam. DOI:10. 6041/j. issn. 1000-1298. 2014. 02. 047.(in Chinese)
[6]范梦扬,马钦,刘峻明,等.基于机器视觉的大田环境小麦麦穗计数方法[J/OL].农业机械学报,2015,46(增刊):234-239.FAN Mengyang,MA Qin,LIU Junming,et al. Wheat vision counting method for field environment based on machine vision[J/OL]. Transactions of the Chinese Society for Agricultural Machinery,2015,46(Supp.):234-239. http:∥www. j-csam. org/jcsam/ch/reader/view_abstract. aspx? flag=1&file_no=2015S038&journal_id=jcsam. DOI:10. 6041/j. issn. 1000-1298.2015. S0. 038.(in Chinese)
[7]周云成,许童羽,郑伟,等.基于深度卷积神经网络的番茄主要器官分类识别方法[J].农业工程学报,2017,33(15):219-226.ZHOU Yuncheng,XU Tongyu,ZHENG Wei,et al. Classification and recognition method of tomato main organs based on deep convolutional neural network[J]. Transactions of the CSAE,2017,33(15):219-226.(in Chinese)
[8]高震宇,王安,刘勇,等.基于卷积神经网络的鲜茶叶智能分选系统研究[J/OL].农业机械学报,2017,48(7):53-58.GAO Zhenyu,WANG An,LIU Yong,et al. Intelligent fresh-tea-leaves sorting system research based on convolution neural network[J/OL]. Transactions of the Chinese Society for Agricultural Machinery,2017,48(7):53-58. http:∥www. j-csam.org/jcsam/ch/reader/view_abstract. aspx? flag=1&file_no=20170707&journal_id=jcsam. DOI:10. 6041/j. issn. 1000-1298.2017. 07. 007.(in Chinese)
[9] LU Y,YI S,ZENG N,et al. Identification of rice diseases using deep convolutional neural networks[J]. Neurocomputing,2017,267:378-384.
[10] JOHANNES A,PICON A,ALVAREZ-GILA A. Automatic plant disease diagnosis using mobile capture devices,applied on a wheat use case[J]. Computers and Electronics in Agriculture,2017,138:200-209.
[11] LU H,CAO Z,XIAO Y,et al. Tassel Net:counting maize tassels in the wild via local counts regression network[J]. Plant Methods,2017,13:79-95.
[12] UBBENS J,CIESLAK M,PRUSINKIEWICZ P,et al. The use of plant models in deep learning:an application to leaf counting in rosette plants[J]. Plant Methods,2018,14:6-15.
[13] TANG J,WANG D,ZHANG Z,et al. Weed identification based on K-means feature learning combined with convolutional neural network[J]. Computers and Electronics in Agriculture,2017,135:63-70.
[14] FERREIRA A D S,FREITAS D M,SILVA G G D,et al. Weed detection in soybean crops using Conv Nets[J]. Computers and Electronics in Agriculture,2017,143:314-324.
[15] GHAZI M M,YANIKOGLU B,APTOULA E. Plant identification using deep neural networks via optimization of transfer learning parameters[J]. Neurocomputing,2017,235:228-235.
[16] DYRMANN M,KARSTOFT H,MIDTIBY H S. Plant species classification using deep convolutional neural network[J].Biosystems Engineering,2016,151:72-80.
[17] RUβWURM M,KORNER M. Multi-temporal land cover classification with long short-term memory neural networks[J].ISPRS—International Archives of the Photogrammetry,Remote Sensing and Spatial Information Sciences,2017,42:551-558.
[18] KAMILARIS A,PRENAFETA-BOLDúF X. Deep learning in agriculture:a survey[J]. Computers and Electronics in Agriculture,2018,147:70-90.
[19] XIONG X,DUAN L,LIU L,et al. Panicle-SEG:a robust image segmentation method for rice panicles in the field based on deep learning and superpixel optimization[J]. Plant Methods,2017,13:104-118.
[20]马浚诚,温皓杰,李鑫星,等.基于图像处理的温室黄瓜霜霉病诊断系统[J/OL].农业机械学报,2017,48(2):195-202.MA Juncheng,WEN Haojie,LI Xinxing,et al. Greenhouse cucumber downy mildew diagnosis system based on image processing[J/OL]. Transactions of the Chinese Society for Agricultural Machinery,2017,48(2):195-202. http:∥www. jcsam. org/jcsam/ch/reader/view_abstract. aspx? flag=1&file_no=20170226&journal_id=jcsam. DOI:10. 6041/j. issn.1000-1298. 2017. 02. 026.(in Chinese)
[21]马浚诚,李鑫星,温皓杰,等.面向叶类蔬菜病害识别的温室监控视频采集系统[J/OL].农业机械学报,2015,46(3):282-287.MA Juncheng,LI Xinxing,WEN Haojie,et al. Monitoring video capture system for identification of greenhouse vegetable diseases[J/OL]. Transactions of the Chinese Society for Agricultural Machinery,2015,46(3):282-287. http:∥www. jcsam. org/jcsam/ch/reader/view_abstract. aspx? flag=1&file_no=20150341&journal_id=jcsam. DOI:10. 6041/j. issn.1000-1298. 2015. 03. 041.(in Chinese)
[22]马浚诚,杜克明,郑飞翔,等.基于卷积神经网络的温室黄瓜病害识别系统[J].农业工程学报,2018,34(12):186-192.MA Juncheng, DU Keming, ZHENG Feixiang, et al. Greenhouse cucumber disease identification system based on convolutional neural network[J]. Transactions of the CSAE,2018,34(12):186-192.(in Chinese)
[23] LECUN Y,BENGIO Y,HINTON G. Deep learning[J]. Nature,2015,521(7553):436-444.
[24] LIU X,LIU W,LIU Y,et al. A survey of deep neural network architectures and their applications[J]. Neurocomputing,2017,234:11-26.
[25] GHOSAL S,BLYSTONE D,SINGH A K,et al. An explainable deep machine vision framework for plant stress phenotyping[J]. Proceedings of the National Academy of Sciences,2018,115(18):4613-4618.
[26] SRIVASTAVA N H G K A. Dropout:a simple way to prevent neural networks from overfitting[J]. Journal of Machine Learning Research,2014,15(1):1929-1958.
[27] HOSANG J,BENENSON R,SCHIELE B. Learning non-maximum suppression[C]∥IEEE Conference on Computer Vision&Pattern Recognition,2017:6469-6477.
[28] BODLA N,SINGH B,CHELLAPPA R,et al. Improving object detection with one line of code[C]∥Proceedings of the IEEE International Conference on Computer Vision,2017:5562-5570.
[29] VEDALDI A,LENC K. MatConvNet:convolutional neural networks for MATLAB[C]∥ACM International Conference on Multimedia,ACM,2015:689-692.
[30] MA J,DU K,ZHENG F,et al. A recognition method for cucumber diseases using leaf symptom images based on deep convolutional neural network[J]. Computers and Electronics in Agriculture,2018,154:18-24.
[31] DING W,TAYLOR G. Automatic moth detection from trap images for pest management[J]. Computers and Electronics in Agriculture,2016,123:17-28.
[32] MA J,LI Y,DU K,et al. Estimating above ground biomass of winter wheat at early growth stages using digital images and deep convolutional neural network[J]. European Journal of Agronomy,2019,103:117-129.