基于CNN的监控视频中人脸图像质量评估
|
摘要
在公共安全领域,监控视频中的人脸识别技术是不可或缺的技术,成为研究热点.而监控视频中低质量的人脸图像会大大降低整个人脸识别系统的识别准确率,系统难以更广泛地被投入实际使用.本文提出了一种基于CNN的人脸图像质量评估方法.通过对Alexnet模型进行改进,将网络中的多个卷积层与全连接层连接,从而提取不同尺度的图像特征.通过端到端的训练过程,预测人脸图像质量分数.另外,采用人脸识别算法来标定人脸图像的质量分数,使质量分数能更有效地筛选出适合识别算法的图像.在Color FERET数据集上实验表明,本文方法能够准确地对人脸图像进行质量评估.而在实际采集的监控视频数据集上实验表明,本文方法能筛选出高质量的人脸图像用作后续人脸识别,提高人脸识别准确率.
Face recognition in surveillance videos is an essential technology in public security and has gotten more and more attention. But it is a little hard for the face recognition systems to be integrated into real application due to the low recognition rate caused partly by low face image quality. This study proposes a method of face image quality assessment using CNN. The proposed net, modified from the Alexnet, connects intermediate convolution layers to fully connect layer,to get multiple image features. Then, face image quality scores can be gotten from proposed net which is trained by end to end. In addition, a face image quality metric is used to relate the quality with the face recognition algorithm. Experiments on Color FERET datasets show that the proposed algorithm is able to elevate the face image quality exactly. Further experiments on a video surveillance dataset(collected by ourselves) show that the proposed method can select high quality face image for face recognition, leading to significant improvements in recognition accuracy.
Face recognition in surveillance videos is an essential technology in public security and has gotten more and more attention. But it is a little hard for the face recognition systems to be integrated into real application due to the low recognition rate caused partly by low face image quality. This study proposes a method of face image quality assessment using CNN. The proposed net, modified from the Alexnet, connects intermediate convolution layers to fully connect layer,to get multiple image features. Then, face image quality scores can be gotten from proposed net which is trained by end to end. In addition, a face image quality metric is used to relate the quality with the face recognition algorithm. Experiments on Color FERET datasets show that the proposed algorithm is able to elevate the face image quality exactly. Further experiments on a video surveillance dataset(collected by ourselves) show that the proposed method can select high quality face image for face recognition, leading to significant improvements in recognition accuracy.
引文
1 Zhao W,Chellappa R,Phillips PJ,et al.Face recognition:A literature survey.ACM Computing Surveys(CSUR),2003,35(4):399-458.[doi:10.1145/954339.954342]
2 Wiskott L,Krüger N,Kuiger N,et al.Face recognition by elastic bunch graph matching.IEEE Transactions on Pattern Analysis and Machine Intelligence,1997,19(7):775-779.[doi:10.1109/34.598235]
3 Blanz V,Vetter T.Face recognition based on fitting a 3D morphable model.IEEE Transactions on Pattern Analysis and Machine Intelligence,2003,25(9):1063-1074.[doi:10.1109/TPAMI.2003.1227983]
4 Wright J,Yang AY,Ganesh A,et al.Robust face recognition via sparse representation.IEEE Transactions on Pattern Analysis and Machine Intelligence,2009,31(2):210-227.[doi:10.1109/TPAMI.2008.79]
5 Grother P,Tabassi E.Performance of biometric quality measures.IEEE Transactions on Pattern Analysis and Machine Intelligence,2007,29(4):531-543.[doi:10.1109/TPAMI.2007.1019]
6 Grother PJ,Quinn GW,Phillips PJ.Report on the evaluation of 2D still-image face recognition algorithms.NISTInteragency Report,2010,7709:106.[doi:10.6028/NIST.IR.7709]
7 Huang GB,Ramesh M,Berg T,et al.Labeled faces in the wild:A database for studying face recognition in unconstrained environments.University of Massachusetts Amherst Technical Report.Amherst,Massachusetts:University of Massachusetts Amherst,2007.
8 Fan HQ,Cao ZM,Jiang YN,et al.Learning deep face representation.arXiv preprint arXiv:1403.2802,2014.
9魏政刚,袁杰辉,蔡元龙.图象质量评价方法的历史、现状和未来.中国图象图形学报,1998,3(5):386-389.[doi:10.3969/j.issn.1006-8961.1998.05.008]
10 Sang JT,Lei Z,Li SZ.Face image quality evaluation for ISO/IEC standards 19794-5 and 29794-5.Proceedings of the3rd International Conference on Advances in Biometrics.Alghero,Italy.2009.229-238.
11 Nasrollahi K,Moeslund TB.Face quality assessment system in video sequences.Proceedings of the First European Workshop on Biometrics and Identity Management.Roskilde,Denmark.2008.10-18.
12 Rúa EA,Castro JLA,Mateo CG.Quality-based score normalization and frame selection for video-based person authentication.Proceedings of the First European Workshop on Biometrics and Identity Management.Roskilde,Denmark.2008.1-9.
13蒋刚毅,黄大江,王旭,等.图像质量评价方法研究进展.电子与信息学报,2010,32(1):219-226.
14邹国锋,傅桂霞,李震梅,等.融合二级评价指标的人脸图像质量评价方法.山东大学学报(工学版),2016,46(2):6-13.
15 Ozay N,Tong Y,Wheeler FW,et al.Improving face recognition with a quality-based probabilistic framework.Proceedings of 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.Boston,MA,USA.2009.134-141.
16 Chen JS,Deng Y,Bai GC,et al.Face image quality assessment based on learning to rank.IEEE Signal Processing Letters,2015,22(1):90-94.[doi:10.1109/LSP.2014.2347419]
17 Kang L,Ye P,Li Y,et al.Convolutional neural networks for no-reference image quality assessment.Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition.Columbus,OH,USA.2014.1733-1740.
18 Liu GR,Xu Y,Lan JP.No-reference face image assessment based on deep features.Proceedings Volume 9971,Applications of Digital Image Processing XXXIX.San Diego,CA,USA.2016.9971:99711S.
19 Krizhevsky A,Sutskever I,Hinton GE.ImageNet classification with deep convolutional neural networks. Proceedings of the 25th International Conference on Neural Information Processing Systems.Lake Tahoe,NV,USA.2012.1097-1105.[doi:10.1145/3065386]
20 Hariharan B,Arbeláez P,Girshick R,et al.Hypercolumns for object segmentation and fine-grained localization.Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston,MA,USA.2015.447-456.
21 Parkhi OM,Vedaldi A,Zisserman A.Deep face recognition.Proceedings of British Machine Vision Conference.Swansea,UK.2015.6.
22 Phillips PJ,Moon H,Rizvi SA,et al.The FERET evaluation methodology for face-recognition algorithms.IEEE Transactions on Pattern Analysis and Machine Intelligence,2000,22(10):1090-1104.[doi:10.1109/34.879790]
23 Sim T,Baker S,Bsat M.The CMU pose,illumination,and expression database.IEEE Transactions on Pattern Analysis and Machine Intelligence,2003,25(12):1615-1618.
24 Pan CH,Ni BB,Xu Y,et al.Recognition oriented facial image quality assessment via deep convolutional neural network.Proceedings of the International Conference on Internet Multimedia Computing and Service.Xi’an,China.2016.160-163.
2 Wiskott L,Krüger N,Kuiger N,et al.Face recognition by elastic bunch graph matching.IEEE Transactions on Pattern Analysis and Machine Intelligence,1997,19(7):775-779.[doi:10.1109/34.598235]
3 Blanz V,Vetter T.Face recognition based on fitting a 3D morphable model.IEEE Transactions on Pattern Analysis and Machine Intelligence,2003,25(9):1063-1074.[doi:10.1109/TPAMI.2003.1227983]
4 Wright J,Yang AY,Ganesh A,et al.Robust face recognition via sparse representation.IEEE Transactions on Pattern Analysis and Machine Intelligence,2009,31(2):210-227.[doi:10.1109/TPAMI.2008.79]
5 Grother P,Tabassi E.Performance of biometric quality measures.IEEE Transactions on Pattern Analysis and Machine Intelligence,2007,29(4):531-543.[doi:10.1109/TPAMI.2007.1019]
6 Grother PJ,Quinn GW,Phillips PJ.Report on the evaluation of 2D still-image face recognition algorithms.NISTInteragency Report,2010,7709:106.[doi:10.6028/NIST.IR.7709]
7 Huang GB,Ramesh M,Berg T,et al.Labeled faces in the wild:A database for studying face recognition in unconstrained environments.University of Massachusetts Amherst Technical Report.Amherst,Massachusetts:University of Massachusetts Amherst,2007.
8 Fan HQ,Cao ZM,Jiang YN,et al.Learning deep face representation.arXiv preprint arXiv:1403.2802,2014.
9魏政刚,袁杰辉,蔡元龙.图象质量评价方法的历史、现状和未来.中国图象图形学报,1998,3(5):386-389.[doi:10.3969/j.issn.1006-8961.1998.05.008]
10 Sang JT,Lei Z,Li SZ.Face image quality evaluation for ISO/IEC standards 19794-5 and 29794-5.Proceedings of the3rd International Conference on Advances in Biometrics.Alghero,Italy.2009.229-238.
11 Nasrollahi K,Moeslund TB.Face quality assessment system in video sequences.Proceedings of the First European Workshop on Biometrics and Identity Management.Roskilde,Denmark.2008.10-18.
12 Rúa EA,Castro JLA,Mateo CG.Quality-based score normalization and frame selection for video-based person authentication.Proceedings of the First European Workshop on Biometrics and Identity Management.Roskilde,Denmark.2008.1-9.
13蒋刚毅,黄大江,王旭,等.图像质量评价方法研究进展.电子与信息学报,2010,32(1):219-226.
14邹国锋,傅桂霞,李震梅,等.融合二级评价指标的人脸图像质量评价方法.山东大学学报(工学版),2016,46(2):6-13.
15 Ozay N,Tong Y,Wheeler FW,et al.Improving face recognition with a quality-based probabilistic framework.Proceedings of 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.Boston,MA,USA.2009.134-141.
16 Chen JS,Deng Y,Bai GC,et al.Face image quality assessment based on learning to rank.IEEE Signal Processing Letters,2015,22(1):90-94.[doi:10.1109/LSP.2014.2347419]
17 Kang L,Ye P,Li Y,et al.Convolutional neural networks for no-reference image quality assessment.Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition.Columbus,OH,USA.2014.1733-1740.
18 Liu GR,Xu Y,Lan JP.No-reference face image assessment based on deep features.Proceedings Volume 9971,Applications of Digital Image Processing XXXIX.San Diego,CA,USA.2016.9971:99711S.
19 Krizhevsky A,Sutskever I,Hinton GE.ImageNet classification with deep convolutional neural networks. Proceedings of the 25th International Conference on Neural Information Processing Systems.Lake Tahoe,NV,USA.2012.1097-1105.[doi:10.1145/3065386]
20 Hariharan B,Arbeláez P,Girshick R,et al.Hypercolumns for object segmentation and fine-grained localization.Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston,MA,USA.2015.447-456.
21 Parkhi OM,Vedaldi A,Zisserman A.Deep face recognition.Proceedings of British Machine Vision Conference.Swansea,UK.2015.6.
22 Phillips PJ,Moon H,Rizvi SA,et al.The FERET evaluation methodology for face-recognition algorithms.IEEE Transactions on Pattern Analysis and Machine Intelligence,2000,22(10):1090-1104.[doi:10.1109/34.879790]
23 Sim T,Baker S,Bsat M.The CMU pose,illumination,and expression database.IEEE Transactions on Pattern Analysis and Machine Intelligence,2003,25(12):1615-1618.
24 Pan CH,Ni BB,Xu Y,et al.Recognition oriented facial image quality assessment via deep convolutional neural network.Proceedings of the International Conference on Internet Multimedia Computing and Service.Xi’an,China.2016.160-163.