深度残差神经网络高分辨率遥感图像建筑物分割
|
摘要
针对高分辨率遥感图像建筑物分割问题,提出一种Encoder-Decoder的深度学习框架,建立输入图像到分割结果之间的端对端的分割模型。其中Encoder以残差网络为基础,自动提取建筑物的特征;Decoder采用反卷积实现对特征图的上采样,从而完成对建筑物的分割;同时引入批量规范化处理,降低了神经网络权重训练过程中的梯度竞争,从而减小了神经网络的训练难度。实验表明:提出的建筑物分割算法能有效提取建筑物的块状特征和边缘信息,降低复杂道路等干扰的影响,提升建筑物的分割精准度,算法对邻近复杂道路的建筑物、规律性建筑物、单体复杂建筑物等3种典型建筑物的分割精度分别为:0.837、0.892和0.630;F值分别为:0.851、0.879和0.730。同时,多分辨率条件下的分割实验结果表明,该算法对于一定范围内的多分辨率遥感图像具有较好的泛化能力。
This paper addresses the buildings segmentation in high resolution remote sensing image and proposes an Encoder-Decoder architecture of deep learning with End-to-End model,in which Encoder is based on ResNet,and the features needed by segmentation are exacted automatically,and the Decoder produces the segmentation result by deconvolution. Furthermore,in the training process,batch normalization is employed to decrease the gradient competition,so as to reduce the difficulty of training the deep neural network.The experiment results show that the algorithm effectively exacts the bulk feature and edge information of building in the high resolution remote sensing image,therefore the complex road disturbance is suppressed convincingly,and the building segmentation precision is improved effectively,the segmentation precision for three typical buildings,the building besides complex road,the ordered buildings and the complex single building,are 0.836 5,0.892 4,and 0.629 7 respectively;and the F-measure are 0.851 4,0.878 6 and 0.729 8,respectively. Meanwhile,the experiment results for multi-resolution remote sensing images show that the method can be generalized to the multi-resolution image within limits.
This paper addresses the buildings segmentation in high resolution remote sensing image and proposes an Encoder-Decoder architecture of deep learning with End-to-End model,in which Encoder is based on ResNet,and the features needed by segmentation are exacted automatically,and the Decoder produces the segmentation result by deconvolution. Furthermore,in the training process,batch normalization is employed to decrease the gradient competition,so as to reduce the difficulty of training the deep neural network.The experiment results show that the algorithm effectively exacts the bulk feature and edge information of building in the high resolution remote sensing image,therefore the complex road disturbance is suppressed convincingly,and the building segmentation precision is improved effectively,the segmentation precision for three typical buildings,the building besides complex road,the ordered buildings and the complex single building,are 0.836 5,0.892 4,and 0.629 7 respectively;and the F-measure are 0.851 4,0.878 6 and 0.729 8,respectively. Meanwhile,the experiment results for multi-resolution remote sensing images show that the method can be generalized to the multi-resolution image within limits.
引文
[1] Chen Jie,Deng Min,Xiao Pengfeng,et al. Object-oriented Classification of High Resolution Imagery based on Watershed Transform and Sptial Clustering[J]. Remote Sensing Technology and Application,2010,25(5):597-603.[陈杰,邓敏,肖鹏峰,等.基于分水岭变换与空间聚类的高分辨率遥感影像面向对象分类[J].遥感技术与应用,2010,25(5):597-603.]
[2] Wang Yu,Wang Baoshan,Wang Tian,et al. Image Entropy Active Contour Models towards Water Area Segmentation in Remote Sensing Image[J]. Optics and Precision Engineering,2018[王宇,王宝山,王田,等.面向遥感图像水域分割的图像熵主动轮廓模型[J].光学精密工程,2018,26(3):698-707.]
[3] Wu Jun,Wang Yuanyuan,Chen Yue,et al. Speckle Reduction of Ultrasound Images with Anisotropic Diffusion based on Homogeneous Region Automatic Selection[J]. Optics and Precision Engineering,2014,22(5):1312-1321.[吴俊,汪源源,陈悦,等.基于同质区域自动选取的各向异性扩散超声图像去噪[J].光学精密工程,2014,22(5):1312-1321.]
[4] Chen Kuntang,Dong Xiaolong,Xu Xing’ou,et al. The Study on Oceanic Vector Wind Field Retrieve Technique based on Neural Networks of Microwave Scatterometer[J].Remote Sensing Technology and Application,2017,32(4):683-690.[陈坤堂,董晓龙,徐星欧,等.微波散射计反演海面风场的神经网络方法研究[J].遥感技术与应用,2017,32(4):683-690.]
[5] Wang Yu,Li Yu,Zhao Quanhua. Region-based Multiscale Segmentation of Panchromatic Remote Sensing Image[J].Control and Decision,2018,33(3):535-541.[王玉,李玉,赵泉华.基于区域的多尺度全色遥感图像分割[J].控制与决策,2018,33(3):535-541.]
[6] Hinton G E,Salakhutdinov R R. Reducing the Dimensionality of Data with Neural Networks[J]. Science, 2006, 313(5786):504-507.
[7] Zeng N Y,Zhang H,Song B Y,et al. Facial Expression Recognition via Learning Deep Sparse Autoencoders[J]. Neurocomputing,2018,273(17):643-649.
[8] Xu X Y,Pan J S,Zhang Y J,et al. Motion Blur Kernel Estimation via Deep Learning[J]. IEEE Transactions on Image Processing,2018,27(1):194-205.
[9] Shao H D,Jiang H K,Zhang H Z,et al. Rolling Bearing Fault Feature Learning Using Improved Convolutional Deep Belief Network with Compressed Sensing[J]. Mechanical Systems and Signal Processing,2018,100:743-765.
[10] Long J,Shelhamer E,Darrell T. Fully Convolutional Networks for Semantic Segmentation[C]∥IEEE Transactions on Pattern Analysis&Machine Intelligence,2015:3431-3440.
[11] Shelhamer E,Long J,Darrell T. Fully Convolutional Networks for Semantic Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(4):640-651.
[12] Simonyan K,Zisserman A. Visual Geometry Group[EB/OL]. http:∥www. robots. ox. ac. uk/~vgg/research/very_deep/,2014.
[13] Badrinarayanan V,Kendall A,Cipolla R. SegNet:A Deep Convolutional Encoder-decoder Architecture for Scene Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(12):2481-2495.
[14] Yu F,Koltun V,Multi-scale Context Aggregation by Dilated Convolutions[C]∥International Conference on Learning Representations,2016.
[15] Chen L C,Papandreou G,Kokkinos I,et al. Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs[C]∥International Conference on Learning Representations,2015.
[16] Deeplab-public[EB/OL], https:∥bitbucket. org/deeplab/deeplab-public/,2017,2018.
[17] Vakalopoulou M,Karantzalos K,Komodakis N,et al. Building Detection in very High Resolution Multispectral Data with Deep Learning Features[C]∥IEEE. Geoscience&Remote Sensing Symposium,2015:1873-1876.
[18] Huang Z M,Cheng G L,Wang H Z,et al. Building Extraction from Multi-source Remote Sensing Images via Deep Deconvolution Neural Networks[C]∥IEEE. Geoscience and Remote Sensing Symposium,2016:1835-1838.
[19] Saito S,Aoki Y. Building and Road Detection from Large Aerial Imagery[C]∥Image Processing:Machine Vision Applications VIII,2015:1814-1821.
[20] Yuan J. Learning Building Extraction in Aerial Scenes with Convolutional Networks[J]. IEEE Transactions on Pattern Analysis&Machine Intelligence,2017,40(11):2793-2798.
[21] Bittner K,Cui S Y,Reinartz P. Building Extraction from Remote Sensing Data Using Fully Convolutional Networks[C]∥ISPRS Hannover Workshop:Hrigi,2017:481-486.
[22] Bischke B,Helber P,Folz J,et al. Multi-Task Learning for Segmentation of Building Footprints with Deep Neural Networks[EB/OL].https:∥arxiv.org/abs/1709.05932,2017,2017.
[23] Wang Y,Wang C,Zhang H. Integrating H-A-αwith Fully Convolutional Networks for Fully PolSAR Classification[C]∥IEEE. International Workshop on Remote Sensing with Intelligent Processing,2017:1-4.
[24] Alshehhi R,Marpu P R,Woon W L,et al. Simultaneous Extraction of Roads and Buildings in Remote Sensing Imagery with Convolutional Neural Networks[J]. ISPRS Journal of Photogrammetry and Remote Sensing,2017,130:139-149.
[25] Lin H,Shi Z,Zou Z. Fully Convolutional Network With Task Partitioning for Inshore Ship Detection in Optical Remote Sensing Images[J]. IEEE Geoscience and Remote Sensing Letters,2017,14(10):1665-1669.
[26] Jiao L,Liang M,Chen H,et al. Deep Fully Convolutional Network-based Spatial Distribution Prediction for Hyperspectral Image Classification[J]. IEEE Transactions on Geoscience and Remote Sensing,2017,55(10):5585-5599.
[27] Ioffe S,Szegedy C. Batch Normalization:Accelerating Deep Network Training by Reducing Internal Covariate Shift[EB/OL],https:∥arxiv.org/abs/1502.03167,2015,2018.
[28] He K M,Zhang X Y,Ren S Q,et al. Deep Residual Learning for Image Recognition[J]. https:∥arxiv. org/abs/1512.03385,2015.
[29] Nirvana Inria Aerial Image Labeling Dataset[EB/OL].https:∥project.inria.fr/aerialimagelabeling/,2016,2017..
[30] Maggiori E,Tarabalka Y,Charpiat G,et al. Can Semantic Labeling Methods Generalize to Any City? The Inria Aerial Image Labeling Benchmark[C]∥IEEE International Symposium on Geoscience and Remote Sensing,2017:3226-3229.
[31] Krizhevsky A,Sutskever I,Hinton G E. ImageNet Classification with Deep Convolutional Neural Networks[C]∥Neural Information Processing Systems Conference,2012:1097-1105
[32] Dumoulin V,Visin F. A Guide to Convolution Arithmetic for Deep Learning[EB/OL], https:∥arxiv. org/abs/1603.07285,2016,2018.
[33] Shelhamer E,Long J,Darrell T. Fully Convolutional Networks for Semantic Segmentation[J]. IEEE Trans Pattern Anal Mach Intell,2017,39(4):640-651.
[2] Wang Yu,Wang Baoshan,Wang Tian,et al. Image Entropy Active Contour Models towards Water Area Segmentation in Remote Sensing Image[J]. Optics and Precision Engineering,2018[王宇,王宝山,王田,等.面向遥感图像水域分割的图像熵主动轮廓模型[J].光学精密工程,2018,26(3):698-707.]
[3] Wu Jun,Wang Yuanyuan,Chen Yue,et al. Speckle Reduction of Ultrasound Images with Anisotropic Diffusion based on Homogeneous Region Automatic Selection[J]. Optics and Precision Engineering,2014,22(5):1312-1321.[吴俊,汪源源,陈悦,等.基于同质区域自动选取的各向异性扩散超声图像去噪[J].光学精密工程,2014,22(5):1312-1321.]
[4] Chen Kuntang,Dong Xiaolong,Xu Xing’ou,et al. The Study on Oceanic Vector Wind Field Retrieve Technique based on Neural Networks of Microwave Scatterometer[J].Remote Sensing Technology and Application,2017,32(4):683-690.[陈坤堂,董晓龙,徐星欧,等.微波散射计反演海面风场的神经网络方法研究[J].遥感技术与应用,2017,32(4):683-690.]
[5] Wang Yu,Li Yu,Zhao Quanhua. Region-based Multiscale Segmentation of Panchromatic Remote Sensing Image[J].Control and Decision,2018,33(3):535-541.[王玉,李玉,赵泉华.基于区域的多尺度全色遥感图像分割[J].控制与决策,2018,33(3):535-541.]
[6] Hinton G E,Salakhutdinov R R. Reducing the Dimensionality of Data with Neural Networks[J]. Science, 2006, 313(5786):504-507.
[7] Zeng N Y,Zhang H,Song B Y,et al. Facial Expression Recognition via Learning Deep Sparse Autoencoders[J]. Neurocomputing,2018,273(17):643-649.
[8] Xu X Y,Pan J S,Zhang Y J,et al. Motion Blur Kernel Estimation via Deep Learning[J]. IEEE Transactions on Image Processing,2018,27(1):194-205.
[9] Shao H D,Jiang H K,Zhang H Z,et al. Rolling Bearing Fault Feature Learning Using Improved Convolutional Deep Belief Network with Compressed Sensing[J]. Mechanical Systems and Signal Processing,2018,100:743-765.
[10] Long J,Shelhamer E,Darrell T. Fully Convolutional Networks for Semantic Segmentation[C]∥IEEE Transactions on Pattern Analysis&Machine Intelligence,2015:3431-3440.
[11] Shelhamer E,Long J,Darrell T. Fully Convolutional Networks for Semantic Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(4):640-651.
[12] Simonyan K,Zisserman A. Visual Geometry Group[EB/OL]. http:∥www. robots. ox. ac. uk/~vgg/research/very_deep/,2014.
[13] Badrinarayanan V,Kendall A,Cipolla R. SegNet:A Deep Convolutional Encoder-decoder Architecture for Scene Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(12):2481-2495.
[14] Yu F,Koltun V,Multi-scale Context Aggregation by Dilated Convolutions[C]∥International Conference on Learning Representations,2016.
[15] Chen L C,Papandreou G,Kokkinos I,et al. Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs[C]∥International Conference on Learning Representations,2015.
[16] Deeplab-public[EB/OL], https:∥bitbucket. org/deeplab/deeplab-public/,2017,2018.
[17] Vakalopoulou M,Karantzalos K,Komodakis N,et al. Building Detection in very High Resolution Multispectral Data with Deep Learning Features[C]∥IEEE. Geoscience&Remote Sensing Symposium,2015:1873-1876.
[18] Huang Z M,Cheng G L,Wang H Z,et al. Building Extraction from Multi-source Remote Sensing Images via Deep Deconvolution Neural Networks[C]∥IEEE. Geoscience and Remote Sensing Symposium,2016:1835-1838.
[19] Saito S,Aoki Y. Building and Road Detection from Large Aerial Imagery[C]∥Image Processing:Machine Vision Applications VIII,2015:1814-1821.
[20] Yuan J. Learning Building Extraction in Aerial Scenes with Convolutional Networks[J]. IEEE Transactions on Pattern Analysis&Machine Intelligence,2017,40(11):2793-2798.
[21] Bittner K,Cui S Y,Reinartz P. Building Extraction from Remote Sensing Data Using Fully Convolutional Networks[C]∥ISPRS Hannover Workshop:Hrigi,2017:481-486.
[22] Bischke B,Helber P,Folz J,et al. Multi-Task Learning for Segmentation of Building Footprints with Deep Neural Networks[EB/OL].https:∥arxiv.org/abs/1709.05932,2017,2017.
[23] Wang Y,Wang C,Zhang H. Integrating H-A-αwith Fully Convolutional Networks for Fully PolSAR Classification[C]∥IEEE. International Workshop on Remote Sensing with Intelligent Processing,2017:1-4.
[24] Alshehhi R,Marpu P R,Woon W L,et al. Simultaneous Extraction of Roads and Buildings in Remote Sensing Imagery with Convolutional Neural Networks[J]. ISPRS Journal of Photogrammetry and Remote Sensing,2017,130:139-149.
[25] Lin H,Shi Z,Zou Z. Fully Convolutional Network With Task Partitioning for Inshore Ship Detection in Optical Remote Sensing Images[J]. IEEE Geoscience and Remote Sensing Letters,2017,14(10):1665-1669.
[26] Jiao L,Liang M,Chen H,et al. Deep Fully Convolutional Network-based Spatial Distribution Prediction for Hyperspectral Image Classification[J]. IEEE Transactions on Geoscience and Remote Sensing,2017,55(10):5585-5599.
[27] Ioffe S,Szegedy C. Batch Normalization:Accelerating Deep Network Training by Reducing Internal Covariate Shift[EB/OL],https:∥arxiv.org/abs/1502.03167,2015,2018.
[28] He K M,Zhang X Y,Ren S Q,et al. Deep Residual Learning for Image Recognition[J]. https:∥arxiv. org/abs/1512.03385,2015.
[29] Nirvana Inria Aerial Image Labeling Dataset[EB/OL].https:∥project.inria.fr/aerialimagelabeling/,2016,2017..
[30] Maggiori E,Tarabalka Y,Charpiat G,et al. Can Semantic Labeling Methods Generalize to Any City? The Inria Aerial Image Labeling Benchmark[C]∥IEEE International Symposium on Geoscience and Remote Sensing,2017:3226-3229.
[31] Krizhevsky A,Sutskever I,Hinton G E. ImageNet Classification with Deep Convolutional Neural Networks[C]∥Neural Information Processing Systems Conference,2012:1097-1105
[32] Dumoulin V,Visin F. A Guide to Convolution Arithmetic for Deep Learning[EB/OL], https:∥arxiv. org/abs/1603.07285,2016,2018.
[33] Shelhamer E,Long J,Darrell T. Fully Convolutional Networks for Semantic Segmentation[J]. IEEE Trans Pattern Anal Mach Intell,2017,39(4):640-651.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |