Tetrolet框架下红外与可见光图像融合
|
摘要
提出一种Tetrolet框架下基于联合稀疏表示结合改进脉冲耦合神经网络规则的红外与可见光图像融合方法.对源红外与可见光图像进行不考虑旋转和反射情况下的Tetrolet系数分解;采用联合稀疏方法进行低频系数融合,通过学习字典进行低频系数的精确拟合并融合.在高频子带系数融合上,采用改进脉冲耦合神经网络设置相应的融合规则,根据神经元的点火次数来选择融合图像的高频系数;并对处理后的高低频系数值进行Tetrolet逆变换获取最终融合结果.结果表明,该方法能够有效保留待融合图像的边缘与细节特征,融合结果具有良好的视觉效果,能够增强观察者对于场景的感知和重要目标的识别能力.在互信息、梯度信息、结构相似度以及视觉敏感度指标上都优于传统变换域融合方法,尤其在结构相似度以及梯度保持度上分别领先0.033和0.025,具有有效性.
An fusion method based on joint sparse representation and improved pulse coupled neural network in tetrolet framework was proposed.The original infrared and visible images were decomposed without considering the rotation and reflection;for the low-frequency sub-band coefficients,the joint sparse representation method was used to accurately fit and fuse the low-frequency coefficients through the learning dictionary.In the high frequency subband coefficient fusion,the corresponding fusion rule was set by using the improved pulse coupled neural network,and the high frequency coefficient of the fused image was selected according to the number of firings of the neuron.The processed coefficient values were inversely transformed by tetrolet frame to obtain the final fusion result.The results show that the proposed method can effectively preserve the edge and detail features of the image to be fused,and the fusion results have better visual effects,which can enhance the observer′s ability to perceive the scene and identify important targets.It is superior to the traditional transform domain fusion method in mutual information,gradient information,structural similarity and visual sensitivity index,especially in terms of structural similarity and gradient retention,leading by 0.033 and 0.025,respectively,and has effectiveness.
An fusion method based on joint sparse representation and improved pulse coupled neural network in tetrolet framework was proposed.The original infrared and visible images were decomposed without considering the rotation and reflection;for the low-frequency sub-band coefficients,the joint sparse representation method was used to accurately fit and fuse the low-frequency coefficients through the learning dictionary.In the high frequency subband coefficient fusion,the corresponding fusion rule was set by using the improved pulse coupled neural network,and the high frequency coefficient of the fused image was selected according to the number of firings of the neuron.The processed coefficient values were inversely transformed by tetrolet frame to obtain the final fusion result.The results show that the proposed method can effectively preserve the edge and detail features of the image to be fused,and the fusion results have better visual effects,which can enhance the observer′s ability to perceive the scene and identify important targets.It is superior to the traditional transform domain fusion method in mutual information,gradient information,structural similarity and visual sensitivity index,especially in terms of structural similarity and gradient retention,leading by 0.033 and 0.025,respectively,and has effectiveness.
引文
[1]DING Wen-shan,BI Du-yan,HE Lin-yuan.Infrared and visible image fusion method based on sparse features[J].Infrared Physics&Technology,2018,92(8):372-380.
[2]MA Jia-yi,MA Yong,LI Chang.Infrared and visible image fusion methods and applications:A survey[J].Information Fusion,2018,45(1):153-178.
[3]LIU Yu,CHEN Xun,WANG Zeng-fu.Deep learning for pixel-level image fusion:Recent advances and future prospects[J].Information Fusion,2018,42(7):158-173.
[4]CAI Huai-yu,ZHUO Li-ran,ZHU Pan.Fusion of infrared and visible images based on non-subsampled contourlet transform and intuitionistic fuzzy set[J].Acta Photonica Sinica,2018,47(6):225-234.蔡怀宇,卓励然,朱攀.基于非下采样轮廓波变换和直觉模糊集的红外与可见光图像融合[J].光子学报,2018,47(6):225-234.
[5]FENG Xin,ZHANG Jian-hua,HU Kai-qun.The infrared and visible image fusion method based on variational multiscale[J].Acta Electronica Sinica,2018,46(3):680-687.冯鑫,张建华,胡开群.基于变分多尺度的红外与可见光图像融合[J].电子学报,2018,46(3):680-687.
[6]LIU Xian-hong,CHEN Zhi-bin,QIN Meng-ze.Infrared and visible image fusion using guided filter and convolutional sparse representation[J].Optics and Precision Engineering,2018,26(5):1242-1253.刘先红,陈志斌,秦梦泽.结合引导滤波和卷积稀疏表示的红外与可见光图像融合[J].光学精密工程,2018,26(5):1242-1253.
[7]ZHU Hao-ran,LIU Yun-qing,ZHANG Wen-ying.Infrared and visible image fusion based on contrast enhancement and multiscale edge-preserving decomposition[J].Journal of Electronics&Information Technology,2018,40(6):1294-1300.朱浩然,刘云清,张文颖.基于对比度增强与多尺度边缘保持分解的红外与可见光图像融合[J].电子与信息学报,2018,40(6):1294-1300.
[8]JIANG Ze-tao,WU Hui,ZHOU Xiao-ling.Infrared and visible image fusion algorithm based on improved guided filtering and dual-channel spiking cortical model[J].Acta Optica Sinica,2018,38(2):119-127.江泽涛,吴辉,周哓玲.基于改进引导滤波和双通道脉冲发放皮层模型的红外与可见光图像融合算法[J].光学学报,2018,38(2):119-127.
[9]DENG Li-nuan,YAO Xin-feng.Research on the fusion algorithm of infrared and visible images based on non-subsampled shearlet transform[J].Acta Electronica Sinica,2017,45(12):2965-2970.邓立暖,尧新峰.基于NSST的红外与可见光图像融合算法[J].电子学报,2017,45(12):2965-2970.
[10]DING Wen-shen,BI Du-yan,HE Lin-yuan.Fusion of infrared and visible images based on shearlet transform and neighborhood structure features[J].Acta Optica Sinica,2017,37(10):115-123.丁文杉,毕笃彦,何林远.基于剪切波变换和邻域结构特征的红外与可见光图像融合[J].光学学报,2017,37(10):115-123.
[11]JIANG Yong,WANG Ming-hui.Image fusion with morphological component analysis[J].Information Fusion,2014,18(7):107-118.
[12]KROMMWEH J.Tetrolet transform:a new adaptive Haar wavelet algorithm for sparse image representation[J].Journal of Visual Communication and Image Representation,2010,21(4):364-374.
[13]YAN Xiang,QIN Han-lin,LIU Shang-qian.Image fusion based on Tetrolet transform[J].Journal of Optoelectronics Laser,2013,24(8):1629-1633.延翔,秦翰林,刘上乾.基于Tetrolet变换的图像融合[J].光电子.激光,2013,24(8):1629-1633.
[14]ZHOU Zhi-qiang,WANG Bo,LI Sun.Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with gaussian and bilateral filters[J].Information Fusion,2016,30(7):15-26.
[15]MA Jin-lei,ZHOU Zhi-qiang,WANG Bo.Infrared and visible image fusion based on visual saliency map and weighted least square optimization[J].Infrared Physics&Technology,2017,82(2):8-17.
[16]SHEN Yu,DANG Jian-wu,FENG Xin.Infrared and visible images fusion based on tetrolet transform[J].Spectroscopy and Spectral Analysis,2013,33(6):1506-1511.沈瑜,党建武,冯鑫.基于Tetrolet变换的红外与可见光融合[J].光谱学与光谱分析,2013,33(6):1506-1511.
[17]REZA H,ADEL M.Multimodal image fusion using sparse representation classification in tetrolet domain[J].Digital Signal Processing,2018,79(8):9-22.
[18]ZHANG Er-lei,ZHANG Xian-grong,JIAO Li-cheng.Spectral-spatial hyper spectral image ensemble classification via joint sparse representation[J].Pattern Recognition,2016,59(11):42-54.
[19]LIU Zheng,ERIK B,XUE Zhi-yun.Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision:a comparative study[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(1):94-109.
[2]MA Jia-yi,MA Yong,LI Chang.Infrared and visible image fusion methods and applications:A survey[J].Information Fusion,2018,45(1):153-178.
[3]LIU Yu,CHEN Xun,WANG Zeng-fu.Deep learning for pixel-level image fusion:Recent advances and future prospects[J].Information Fusion,2018,42(7):158-173.
[4]CAI Huai-yu,ZHUO Li-ran,ZHU Pan.Fusion of infrared and visible images based on non-subsampled contourlet transform and intuitionistic fuzzy set[J].Acta Photonica Sinica,2018,47(6):225-234.蔡怀宇,卓励然,朱攀.基于非下采样轮廓波变换和直觉模糊集的红外与可见光图像融合[J].光子学报,2018,47(6):225-234.
[5]FENG Xin,ZHANG Jian-hua,HU Kai-qun.The infrared and visible image fusion method based on variational multiscale[J].Acta Electronica Sinica,2018,46(3):680-687.冯鑫,张建华,胡开群.基于变分多尺度的红外与可见光图像融合[J].电子学报,2018,46(3):680-687.
[6]LIU Xian-hong,CHEN Zhi-bin,QIN Meng-ze.Infrared and visible image fusion using guided filter and convolutional sparse representation[J].Optics and Precision Engineering,2018,26(5):1242-1253.刘先红,陈志斌,秦梦泽.结合引导滤波和卷积稀疏表示的红外与可见光图像融合[J].光学精密工程,2018,26(5):1242-1253.
[7]ZHU Hao-ran,LIU Yun-qing,ZHANG Wen-ying.Infrared and visible image fusion based on contrast enhancement and multiscale edge-preserving decomposition[J].Journal of Electronics&Information Technology,2018,40(6):1294-1300.朱浩然,刘云清,张文颖.基于对比度增强与多尺度边缘保持分解的红外与可见光图像融合[J].电子与信息学报,2018,40(6):1294-1300.
[8]JIANG Ze-tao,WU Hui,ZHOU Xiao-ling.Infrared and visible image fusion algorithm based on improved guided filtering and dual-channel spiking cortical model[J].Acta Optica Sinica,2018,38(2):119-127.江泽涛,吴辉,周哓玲.基于改进引导滤波和双通道脉冲发放皮层模型的红外与可见光图像融合算法[J].光学学报,2018,38(2):119-127.
[9]DENG Li-nuan,YAO Xin-feng.Research on the fusion algorithm of infrared and visible images based on non-subsampled shearlet transform[J].Acta Electronica Sinica,2017,45(12):2965-2970.邓立暖,尧新峰.基于NSST的红外与可见光图像融合算法[J].电子学报,2017,45(12):2965-2970.
[10]DING Wen-shen,BI Du-yan,HE Lin-yuan.Fusion of infrared and visible images based on shearlet transform and neighborhood structure features[J].Acta Optica Sinica,2017,37(10):115-123.丁文杉,毕笃彦,何林远.基于剪切波变换和邻域结构特征的红外与可见光图像融合[J].光学学报,2017,37(10):115-123.
[11]JIANG Yong,WANG Ming-hui.Image fusion with morphological component analysis[J].Information Fusion,2014,18(7):107-118.
[12]KROMMWEH J.Tetrolet transform:a new adaptive Haar wavelet algorithm for sparse image representation[J].Journal of Visual Communication and Image Representation,2010,21(4):364-374.
[13]YAN Xiang,QIN Han-lin,LIU Shang-qian.Image fusion based on Tetrolet transform[J].Journal of Optoelectronics Laser,2013,24(8):1629-1633.延翔,秦翰林,刘上乾.基于Tetrolet变换的图像融合[J].光电子.激光,2013,24(8):1629-1633.
[14]ZHOU Zhi-qiang,WANG Bo,LI Sun.Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with gaussian and bilateral filters[J].Information Fusion,2016,30(7):15-26.
[15]MA Jin-lei,ZHOU Zhi-qiang,WANG Bo.Infrared and visible image fusion based on visual saliency map and weighted least square optimization[J].Infrared Physics&Technology,2017,82(2):8-17.
[16]SHEN Yu,DANG Jian-wu,FENG Xin.Infrared and visible images fusion based on tetrolet transform[J].Spectroscopy and Spectral Analysis,2013,33(6):1506-1511.沈瑜,党建武,冯鑫.基于Tetrolet变换的红外与可见光融合[J].光谱学与光谱分析,2013,33(6):1506-1511.
[17]REZA H,ADEL M.Multimodal image fusion using sparse representation classification in tetrolet domain[J].Digital Signal Processing,2018,79(8):9-22.
[18]ZHANG Er-lei,ZHANG Xian-grong,JIAO Li-cheng.Spectral-spatial hyper spectral image ensemble classification via joint sparse representation[J].Pattern Recognition,2016,59(11):42-54.
[19]LIU Zheng,ERIK B,XUE Zhi-yun.Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision:a comparative study[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2012,34(1):94-109.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |