基于SURF和SIFT特征的视频镜头分割算法
|
摘要
在视频镜头分割处理中,SIFT(Scale-invariant feature transform)特征由于其具有尺度、旋转不变性等诸多优点而被广泛应用,但是SIFT特征提取复杂,计算量大,导致程序效率低下,很难满足实时性的要求。本文提出了一种基于SURF(Speeded Up Robust Features)和SIFT特征的视频镜头分割算法。首先对相关帧进行颜色直方图特征提取,然后通过一个自适应阈值判断是否为候选帧,如果不是,则融合SURF特征再进行复检。最后筛选出的候选帧再通过SIFT算法进行复选,去除误检帧,有效解决了SIFT特征低效率的问题。实验结果表明,对于切变镜头查全率为95%,查准率为93%,对于渐变镜头查全率为90%,查准率为76%,平均单帧运行时间为0.26s,本算法与SIFT、SURF等单一特征的算法相比,在高效性和准确性之间取得了平衡,并且它是鲁棒的。
In the field of video shot segmentation processing,SIFT features are widely used due to their many advantages such as scale and rotation invariance.However,SIFT feature extraction is complex and computationally intensive,resulting in low program efficiency and hard to achieve the requirement of real-time.In this case,an algorithm based on SURF features and SIFT features is proposed in this paper.The algorithm first performs color histogram feature extraction on the frame,determines whether it is a candidate frame through an adaptive threshold,and if not,fuses the SURF feature for re-detection.The candidate frames that are filtered out are then checked by the SIFT algorithm.This can effectively solve the problem of SIFT feature inefficiency.Experimental results show that the recall rate is 95% for the shear shots,93% for the precision,90% for the gradual shots,76% for the precision and 0.26 sfor the average single frame running time and compared with SIFT,SURF and other algorithms with a single feature,this algorithm achieves a balance between efficiency and accuracy,and has better robustness.
In the field of video shot segmentation processing,SIFT features are widely used due to their many advantages such as scale and rotation invariance.However,SIFT feature extraction is complex and computationally intensive,resulting in low program efficiency and hard to achieve the requirement of real-time.In this case,an algorithm based on SURF features and SIFT features is proposed in this paper.The algorithm first performs color histogram feature extraction on the frame,determines whether it is a candidate frame through an adaptive threshold,and if not,fuses the SURF feature for re-detection.The candidate frames that are filtered out are then checked by the SIFT algorithm.This can effectively solve the problem of SIFT feature inefficiency.Experimental results show that the recall rate is 95% for the shear shots,93% for the precision,90% for the gradual shots,76% for the precision and 0.26 sfor the average single frame running time and compared with SIFT,SURF and other algorithms with a single feature,this algorithm achieves a balance between efficiency and accuracy,and has better robustness.
引文
[1]来毅,辛可嘉,刘颖.多特征融合的视频镜头分割[J].电讯技术,2018,58(7):792-797.LAI Y,XIN K J,LIU Y.Video shot segmentation based on multi-feature fusion[J].Telecommunication Engineering,2018,58(7):792-797.(in Chinese)
[2]KIKUKAWA T,KAWAFUCHI S.Development of an automatic summary editing system for the audio visual resources[J].Trans on Electronics and Information.1992.75:204-212.
[3]ZHANG H J,KANKANHALLI A,SMOLIAR S W.Smoliar.Automatic partitioning of full-motion video[J].Multimedia Systems,1993,1(1):10-28.
[4]杨瑞琴,吕进来.基于双重检测的视频镜头分割方法[J].计算机工程与设计,2018,39(5):1393-1398.YANG R Q,LV J L.Video shot segmentation method based on double detection[J].Computer Engineering and Design.2018,39(5):1393-1398.(in Chinese)
[5]于俊清,张强,王赠凯,何云峰.利用回放场景和情感激励检测足球视频精彩镜头[J].计算机学报,2014,37(6):1268-1280.YU J Q,ZHANG Q,WANG Z K,HE Y F.Soccer highlight detection based on replay and affection arousal model[J].Chinese Journal of Computers,2014,37(6):1268-1280.(in Chinese)
[6]李小琛,黄添强.融合彩色信息与SIFT特征的帧内复制粘贴篡改检测[J].计算机系统应用,2018,27(7):11-18.LI X C,HUANG T Q.Forgery detection of copy-paste video based on fusion of color information and SIFT feature[J].Computer Systems&Applications,2018,27(7):11-18.(in Chinese)
[7]蔡轶珩,胡朝阳,崔益泽.融合颜色信息与特征点的镜头边界检测算法[J].计算机应用,2017,37(S2):95-111.CAI Y H,HU Z Y,CUI Y Z.Shot boundary detection algorithm combining color information and feature points[J].Journal of Computer Applications,2017,37(S2):95-111.(in Chinese)
[8]BAY H,ESS A,TUYTELAARS T.Speeded-up robust features[J].Computer Vision&Image Understanding,2008,110(3):346-359.
[9]冯亦东,孙跃.基于SURF特征提取和FLANN搜索的图像匹配算法[J].图学学报,2015,36(4):650-654.FENG Y D,SUN Y.Image Matching Algorithm Based on SURF Feature Extraction and FLANN Search[J].Journal of Graphics.2015,36(4):650-654.(in Chinese)
[10]任克强,胡梦云.基于改进SURF算子的彩色图像配准算法[J].电子测量与仪器学报,2016,30(5):748-756.REN K Q,HU M Y.Color image registration algorithm based on improved SURF[J].Journal of Electronic Measurement and Instrument.2016,30(5):748-756.(in Chinese)
[11]王蒙军.唇读发声器中视觉信息的检测与处理[D].天津:天津大学,2016.WANG M J.Detecting and processing visual information in speech synthesis system driven by visual-speech[D].Tianjin:Tianjin University.2016.(in Chinese)
[12]JUAN L O.A Comparison of SIFT,PCA-SIFT and SURF[J].International Journal of Image Processing.2009,3:143-152.
[13]LOWE D G.Distinctive image features from scale-invariant keypoints[J].International Journal of Computer Vision,2004:91-110.
[2]KIKUKAWA T,KAWAFUCHI S.Development of an automatic summary editing system for the audio visual resources[J].Trans on Electronics and Information.1992.75:204-212.
[3]ZHANG H J,KANKANHALLI A,SMOLIAR S W.Smoliar.Automatic partitioning of full-motion video[J].Multimedia Systems,1993,1(1):10-28.
[4]杨瑞琴,吕进来.基于双重检测的视频镜头分割方法[J].计算机工程与设计,2018,39(5):1393-1398.YANG R Q,LV J L.Video shot segmentation method based on double detection[J].Computer Engineering and Design.2018,39(5):1393-1398.(in Chinese)
[5]于俊清,张强,王赠凯,何云峰.利用回放场景和情感激励检测足球视频精彩镜头[J].计算机学报,2014,37(6):1268-1280.YU J Q,ZHANG Q,WANG Z K,HE Y F.Soccer highlight detection based on replay and affection arousal model[J].Chinese Journal of Computers,2014,37(6):1268-1280.(in Chinese)
[6]李小琛,黄添强.融合彩色信息与SIFT特征的帧内复制粘贴篡改检测[J].计算机系统应用,2018,27(7):11-18.LI X C,HUANG T Q.Forgery detection of copy-paste video based on fusion of color information and SIFT feature[J].Computer Systems&Applications,2018,27(7):11-18.(in Chinese)
[7]蔡轶珩,胡朝阳,崔益泽.融合颜色信息与特征点的镜头边界检测算法[J].计算机应用,2017,37(S2):95-111.CAI Y H,HU Z Y,CUI Y Z.Shot boundary detection algorithm combining color information and feature points[J].Journal of Computer Applications,2017,37(S2):95-111.(in Chinese)
[8]BAY H,ESS A,TUYTELAARS T.Speeded-up robust features[J].Computer Vision&Image Understanding,2008,110(3):346-359.
[9]冯亦东,孙跃.基于SURF特征提取和FLANN搜索的图像匹配算法[J].图学学报,2015,36(4):650-654.FENG Y D,SUN Y.Image Matching Algorithm Based on SURF Feature Extraction and FLANN Search[J].Journal of Graphics.2015,36(4):650-654.(in Chinese)
[10]任克强,胡梦云.基于改进SURF算子的彩色图像配准算法[J].电子测量与仪器学报,2016,30(5):748-756.REN K Q,HU M Y.Color image registration algorithm based on improved SURF[J].Journal of Electronic Measurement and Instrument.2016,30(5):748-756.(in Chinese)
[11]王蒙军.唇读发声器中视觉信息的检测与处理[D].天津:天津大学,2016.WANG M J.Detecting and processing visual information in speech synthesis system driven by visual-speech[D].Tianjin:Tianjin University.2016.(in Chinese)
[12]JUAN L O.A Comparison of SIFT,PCA-SIFT and SURF[J].International Journal of Image Processing.2009,3:143-152.
[13]LOWE D G.Distinctive image features from scale-invariant keypoints[J].International Journal of Computer Vision,2004:91-110.