基于最强引力的空间非对齐图像数据风格迁移
|
摘要
针对图像风格迁移(Style Transfer)任务中的空间非对齐图像数据处理效果不理想的问题,提出一种基于语义学的最强引力方法.该方法是将图片与目标图片看作是一些高维度特征点(High-dimensional Feature Points)的集合,通过定义引力(Gravitation)来衡量两张图片中高维度特征点的相似程度.如果两张图片相似,则对应高维度特征点也互相吸引.生成图像的每个特征点在目标图像中找到与自身引力最强的特征点,然后最小化最强引力损失函数.实验结果表明,该方法对两张图片中语义相似的区域有很强的敏感度,生成图片的质量明显优于若干经典的方法.
In order to solve the problem of spatially non-aligned image data processing in image style transfer tasks,this paper presents a maximum gravitation method based on semantics. This method considers pictures and target pictures as a collection of high-dimensional feature points,and measures the degree of similarity of high-dimensional feature points in two pictures by defining gravitation.If the two pictures are similar,the corresponding high-dimensional feature points also attract each other. Each feature point of the generated image finds the similar feature point in the target image,and then minimizes the maximum gravitation loss function. The experimental results showthat this method has a strong sensitivity to the semantic similarity of two pictures,and the quality of generated pictures is obviously better than some classical methods.
In order to solve the problem of spatially non-aligned image data processing in image style transfer tasks,this paper presents a maximum gravitation method based on semantics. This method considers pictures and target pictures as a collection of high-dimensional feature points,and measures the degree of similarity of high-dimensional feature points in two pictures by defining gravitation.If the two pictures are similar,the corresponding high-dimensional feature points also attract each other. Each feature point of the generated image finds the similar feature point in the target image,and then minimizes the maximum gravitation loss function. The experimental results showthat this method has a strong sensitivity to the semantic similarity of two pictures,and the quality of generated pictures is obviously better than some classical methods.
引文
[1] Deng Ying-ying,Tang Fan,Dong Wei-ming. The status research of stylization of art image[J]. Journal of Nanjing University of Information Technology(Natural Science Edition),2017,9(6):593-598.
[2] Gatys L A,Ecker A S,Bethge M. A neural algorithm of artistic style[J]. Computer Science,2015,111(1):98-136.
[3] Gatys L A,Ecker A S,Bethge M. Image style transfer using convolutional neural netw orks[C]. Computer Vision and Pattern Recognition(CVPR),2016 IEEE Conference on,IEEE,2016:2414-2423.
[4] Wilmot P,Risser E,Barnes C. Stable and controllable neural texture synthesis and style transfer using histogram losses[J]. Computer Science,2017,15(1):96-140.
[5] Li C,Wand M. Combining markov random fields and convolutional neural netw orks for image synthesis[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2016:2479-2486.
[6] Johnson J,Alahi A,Fei-Fei L. Perceptual losses for real-time style transfer and super-resolution[C]. European Conference on Computer Vision,Springer,Cham,2016:694-711.
[7] Cao Jiang-feng. Research and implementation of stylization of image and video based on deep learning[D]. Beijing:Chinese Academy of Sciences University,2017.
[8] Chikontwe P,Lee H J. Towards robust face sketch synthesis with style transfer algorithms[M]. IT Convergence and Security 2017,Springer,Singapore,2018:172-179.
[9] Ignatov A,Kobyshev N,Timofte R,et al. DSLR-quality photos on mobile devices w ith deep convolutional netw orks[C]. Computer Vision and Pattern Recognition(CVPR),2017 IEEE Conference on,IEEE,2017:3297-3305.
[10] Elad M,Milanfar P. Style transfer via texture synthesis[J]. IEEE Transactions on Image Processing,2017,26(5):2338-2351.
[11] Wang X,Oxholm G,Zhang D,et al. Multimodal transfer:a hierarchical deep convolutional neural netw ork for fast artistic style transfer[C]. IEEE Conference on Computer Vision and Pattern Recognition,IEEE Computer Society,2017:7178-7186.
[12] Saleh F,Aliakbarian M S,Salzmann M,et al. Incorporating network built-in priors in w eakly-supervised semantic segmentation[J].IEEE Transactions on Pattern Analysis&M achine Intelligence,2018,40(6):1382-1396.
[1]邓盈盈,唐帆,董未名.图像艺术风格化的研究现状[J].南京信息工程大学学报(自然科学版),2017,9(6):593-598.
[7]操江峰.基于深度学习的图像与视频风格化研究与实现[D].北京:中国科学院大学,2017.
[2] Gatys L A,Ecker A S,Bethge M. A neural algorithm of artistic style[J]. Computer Science,2015,111(1):98-136.
[3] Gatys L A,Ecker A S,Bethge M. Image style transfer using convolutional neural netw orks[C]. Computer Vision and Pattern Recognition(CVPR),2016 IEEE Conference on,IEEE,2016:2414-2423.
[4] Wilmot P,Risser E,Barnes C. Stable and controllable neural texture synthesis and style transfer using histogram losses[J]. Computer Science,2017,15(1):96-140.
[5] Li C,Wand M. Combining markov random fields and convolutional neural netw orks for image synthesis[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2016:2479-2486.
[6] Johnson J,Alahi A,Fei-Fei L. Perceptual losses for real-time style transfer and super-resolution[C]. European Conference on Computer Vision,Springer,Cham,2016:694-711.
[7] Cao Jiang-feng. Research and implementation of stylization of image and video based on deep learning[D]. Beijing:Chinese Academy of Sciences University,2017.
[8] Chikontwe P,Lee H J. Towards robust face sketch synthesis with style transfer algorithms[M]. IT Convergence and Security 2017,Springer,Singapore,2018:172-179.
[9] Ignatov A,Kobyshev N,Timofte R,et al. DSLR-quality photos on mobile devices w ith deep convolutional netw orks[C]. Computer Vision and Pattern Recognition(CVPR),2017 IEEE Conference on,IEEE,2017:3297-3305.
[10] Elad M,Milanfar P. Style transfer via texture synthesis[J]. IEEE Transactions on Image Processing,2017,26(5):2338-2351.
[11] Wang X,Oxholm G,Zhang D,et al. Multimodal transfer:a hierarchical deep convolutional neural netw ork for fast artistic style transfer[C]. IEEE Conference on Computer Vision and Pattern Recognition,IEEE Computer Society,2017:7178-7186.
[12] Saleh F,Aliakbarian M S,Salzmann M,et al. Incorporating network built-in priors in w eakly-supervised semantic segmentation[J].IEEE Transactions on Pattern Analysis&M achine Intelligence,2018,40(6):1382-1396.
[1]邓盈盈,唐帆,董未名.图像艺术风格化的研究现状[J].南京信息工程大学学报(自然科学版),2017,9(6):593-598.
[7]操江峰.基于深度学习的图像与视频风格化研究与实现[D].北京:中国科学院大学,2017.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |