用户名: 密码: 验证码:
基于H.264的时空域误码掩盖技术的研究
详细信息    本馆镜像全文|  推荐本文 |  |   获取CNKI官网全文
摘要
在实时视频传输过程中,数据的丢失总和错误的产生是不可避免,然而少量的传输错误就会导致接收端大量的数据无法正常解码,进而造成视频质量的严重下降。为了解决这一问题,需要采用误码掩盖技术,利用数据在时间和空间上的相关性对接收端的错误信息进行恢复。
     基于客户端的误码掩盖方法能够在恢复图像质量的同时降低网络开销,该方法利用接收端己经正确解码的码流信息通过重建错误区域来改善图像质量。其中,空域掩盖利用丢失宏块与相邻可用宏块信息的相关性通过像素内插来恢复;时域掩盖利用时间冗余信息来估计运动矢量,并通过运动矢量对应的运动补偿宏块替代丢失宏块,然而使用标准的空域和时域误码掩盖算法并不能很好的恢复丢失宏块信息。
     自适应的空域误码掩盖算法(ASEC,Adaptive Spatial Error Concealment)的提出,解决了在现有空域算法不能够很好恢复边缘复杂图像的问题,该算法利用方向信息熵和边缘强度对丢失宏块周围像素点的边缘方向进行分类和判断,并将丢失宏块分为平滑宏块和边缘宏块两种情况:对于平滑宏块,采用双线性插值算法;对于边缘宏块,采用一种新的基于区域分割的方向插值算法。而多参考帧时域误码掩盖算法(MTEC,Multiple-Reference Temporal Error Concealment)的提出,解决了运动矢量预测不够精确的问题,该算法首先确定候选运动矢量集合;然后利用运动估计方法从多个参考帧中获取候选运动补偿宏块,并利用改进的边界匹配法则和搜索算法找到最小边界匹配误差宏块及其对应的运动矢量;最后利用该宏块对丢失宏块进行掩盖。
     实验表明,基于客户端的误码掩盖方法较好的恢复了丢失宏块信息,在提高了图像的解码质量的同时又避免伪边缘的产生。对于多种视频序列和不同网络丢包率,从主观和客观质量上看,ASEC算法和MTEC算法均优于传统的空域和时域误码掩盖算法。
In the process of real-time video transmission, data loss and transmission errors are always inevitable. However, a small amount of errors may lead to incorrect decoding of a large number of codes in client. Furthermore, they may cause serious decline of video quality. To solve this problem, the error concealment technologies are used to recover the damaged information by using the temporal and spatial information redundancy.
     The method of error concealment based on client could not only recover the damaged block but also reduce network overhead. When error occurs during the transmission, receiver uses the correct decoding information to improve image quality via reconstructed damaged block. Among them, spatial concealment recovers damaged block using its neighboring blocks and temporal concealment makes use of temporal information redundancy to estimated motion vector (MV) of the damaged block and this MV is used to conceal the damaged block. But it cannot give satisfying results when using standard spatial and temporal error concealment algorithms.
     For the problems of the existing spatial error concealment algorithms, which could not recover the edge image well, an adaptive spatial error concealment algorithm (ASEC) is proposed. The directional information of the edges extracted from surrounding correctly received blocks are judged by using the directional entropy and edge strength, and the damaged blocks are adaptively classified into uniform blocks and edge blocks. For the uniform blocks, the errors can be concealed by the bilinear interpolation algorithm; for the edge blocks, the errors can be concealed by the directional interpolation algorithm based on region segmentation. Meanwhile, for the problems of the existing spatial error concealment algorithms, a multiple-reference temporal error concealment algorithm (MTEC) is presented to solve the problem of inaccurate of motion vector prediction. Firstly, the candidate motion vector set is determined; then the improved boundary matching algorithm and search algorithm are used to evaluate the candidate motion vectors from a number of reference frames, and the motion vector giving the minimum boundary matching distortion is selected as optimal motion vector; finally it is used to recover the damaged block and the video frame quality can be improved further.
     Results show the error concealment system based on client can commendably recover the data of the damaged block, and it not only improved the quality of the image, but also avoided the false edge. For various video sequences and different macro-block loss ratio, the ASEC and MTEC algorithms are better than traditional spatial and temporal error concealment algorithms from both objectively and subjectively.
引文
[1]刘峰.视频图像编码技术及国际标准.北京:北京邮电大学出版社, 2005. 97~112
    [2] Suh Jae-Won and Ho Yo-Sung. Error Concealment Techniques for Digital TV. IEEE Transactions on Broadcasting, 2002, 48(4):299~306
    [3] Wiegand Thomas, Sullivan Gary J, Bjontegaard Gisle et al. Overview of The H.264/AVC Video Coding Standard. IEEE Transactions on Circuits and System for Video Technology, 2003, 13(7):560~576
    [4]毕厚杰.新一代视频压缩编码标准—H.264/AVC.北京:人民邮电出版社, 2005. 271~279
    [5] Li Tang. Combined and Iterative Form of Spatial and Temporal Error Concealment for Video Signals. IEEE Transactions on Broadcasting, 2006, 52(3):356~361
    [6]万里晴,李学明. H.264的差错控制与误码隐藏技术研究.计算机工程与应用,2007,43(16):53~57
    [7] Gharavi Hamid and Gao Shaohuai. Spatial Interpolation Algorithm for Error Concealment. In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’08), Las Vagas, Nevada, USA, 2008. 1153~1156
    [8] Aign Susanna and Fazel Khaled. Temporal & Spatial Error Concealment Techniques for Hierarchical MPEG-2 Video Codec. In Proceedings of IEEE International Conference on Communications (ICC’95), Seattle, WA, USA, 1995. 1778~1783
    [9] Alkachouh Zaid and Bellanger Maurice G. Fast DCT Based Spatial Domain Interpolation of Block in Images. IEEE Transaction on Image Processing, 2000, 9(4):729~732
    [10] Yu Gong-san and Marcellin Michael W. POCS Based Error Concealment for Packet Video. In Proceedings of Data Compression Conference (DCC’97), Snowbird, Utah, USA. 134~140
    [11] Park Jiho, Park Dong-Chul, Marks Robert J et al. Recovery of Image Blocks Using the Method of Alternating Projections. IEEE Transactions on Image Processing, 2005, 14(4):461~474
    [12] Adisorn Sirikham. Inter-frame Error Concealment Using Pixel Alignment andDirectional Interpolation. In Proceedings of the 11th International Conference on Advanced Communication Technology (ICACT’09), Phoenix Park, Korea, 2009. 1897~1902
    [13] Kim Wonki, Koo Jasung and Jeong Jechang. Fine Directional Interpolation for Spatial Error Concealment. IEEE Transactions on Consumer Electronics, 2006, 52(3):1050~1056
    [14] Kwok Wilson and Sun Huifang. Multi-directional Interpolation for Spatial Error Concealment. IEEE International Conference on Consumer Electronics (ICCE’93), Taipei, Taiwan, China, 1993. 220~221
    [15] Xu Yanling and Zhou Yuanhua. H.264 Video Communication Based Refined Error Concealment Schemes. IEEE Transactions on Consumer Electronics, 2004, 50(4):1135~1141
    [16] Dinh Toan Nguyen, Kien Nguyen Trung and Lee Guee Sang. Interpolation with Sigmoid Functions for Spatial Error Concealment. In Proceedings of the 7th IEEE International Symposium on Signal Processing and Information Technology (ISSPIT’07), Cairo, Egypt, 2007. 961~965
    [17]张荣福,周源华.基于自适应邻域的空域错误隐藏算法.上海交通大学学报, 2004, 38(4):30~33
    [18] Nemethova Nlivia, Ameen Al-Moghrabi and Rupp Markus. Flexible Error Concealment for H.264 Based on Directional Interpolation. In Proceedings of International Conference on Wireless Networks, Communications and Mobile Computing (IWCMC’05), Hawaii, USA, 2005.1255~1260
    [19] Hwa Hyum Seung, Soo Kim Sang, CHul Kim Byoung et al. Efficient Directional Interpolation for Block Recovery Using Difference Values of Border Pixels. In Proceedings of International Congress on Image and Signal Processing (CISP’08), Sanya, China, 2008. 565~ 568
    [20] Zhang Rongfu, Zhou Yuanhua and Huang Xiaodong. Content-Adaptive Spatial Error Concealment for Video Communication. IEEE Transactions on Consumer Electronics, 2004, 50(1):335~341
    [21] Chen Yan, Au Oscar, Zhou Jiantao et al. Adaptively Switching Between Directional Interpolation and Region Matching for Spatial Error Concealment Based on DCT Coefficients. In Proceedings of IEEE International Conference on Multimedia and Expo (ICME’06), New York, USA, 2006. 805~808
    [22] Kuo Ming-Jui, Ting Chi-Cheng and Wang Jung-Hua. Selective Block MatchingAlgorithm for Video System. In Proceedings of IEEE International Conference on Information Reuse and Integration (IRI’04), Las Vagas, Necada, USA, 2004. 163~167
    [23] Lam Wai-Man, Reibman Amy R, Liu Bede. Recovery of Lost or Erroneously Received Motion Vectors. In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP’93), Minneapolis, USA, 1993. 417~420
    [24] Wang Lei, Wang Jun, Goto Satoshi et al. Variable Block Size error Concealment Scheme Based on H.264/AVC Non-normative Decoder. In Proceedings of International Symposium on Intelligent Signal Processing and Communication System (ISPACS’07), Xiamen, China, 2007. 104~107
    [25] Chen Tao, Zhang Ximin and Shi Yun-Qing. Error Concealment Using Refined Boundary Matching Algorithm. In Proceedings of International Conference on Information Technology (CIT’03), Bhubaneswar, India, 2003. 55~59
    [26] Jung Young H, Kim Yong-goo and Choe Yoonsik. Robust Error Concealment Algorithm Using Iterative Weighted Boundary Matching Criterion. In Proceedings of International Conference on Image Processing (ICIP’00), Vancouver, Canada, 2000. 384~387
    [27] Xu Yanling and Zhou Yuanhua. Refined Video Error Concealment Over Wireless IP Network. In Proceedings of the IEEE 6th Symposium on Circuits and Systems Emerging Technologies: Frontiers of Mobile and Wireless Communication (ISCAS’04), Shanghai, China, 2004. 257~260
    [28] Park Chong Soo, Ye Jongchul and Lee Sang Uk. Lost Motion Vector Recovery Algorithm. In Proceedings of IEEE International Symposium on Circuits and Systems (ISCAS’94), London, UK, 1994. 229~232
    [29] Xu Yangling, Zhou Yuanhua. Adaptive Temporal Error Concealment Scheme for H.264/AVC Video Decoder. IEEE Transactions on Consumer Electronics, 2008, 54(4):1846~1851
    [30] Kim Donghyung, Yang Siyoung and Jeong Jechang, A New Temporal Error Concealment Method for H.264 using Adaptive Block Sizes. In Proceedings of IEEE International Conference on Image Processing (ICIP’05), Genoa, Italy, 2005. 628~631
    [31] Al-Mualla Mohammed E, Canagarajah Nishan and Bull David R. Error Concealment Using Motion Field Interpolation. In Proceedings of IEEEInternational Conference on Image Processing (ICIP’98), Chicago, USA, 1998. 512~516
    [32] Mualla Mohammed E, Canagarajah Nishan and Bull David R. Multiple-reference Temporal Error Concealment. In Proceedings of International Symposium on Circuits and Systems (ISCAS’01), Sydney, Australia, 2001. 149~152
    [33] Chen Bangning and Lin Ying. Temporal Error Concealment Using Selective Motion Field Interpolation. Electronics Letters, 2006, 42(24):1390~1391
    [34] Lappalainen Ville, Hallapuro Antti, and Hamalainen Timo D. Optimization of Emerging H.26L Video Encoder. In Proceedings of IEEE Workshop on Signal Processing System (SIPS’01), Antwerp, Belgium, 2001. 406~415
    [35]袁惠瑜. H.264软件编解码器的优化与实现:[硕士学位论文].北京:北京邮电大学图书馆, 2005.
    [36] Agrafiotis Dimitris, Bull David R and Canagarajah Nishan. Spatial Error Concealment with Edge Related Perceptual Considerations. Signal Processing: Image Communication, 2006, 21(2):130~142
    [37] Cafforio Ciro and Fabio Rocca. Method for Measuring Small Displacement of Television Images. IEEE Transactions on Information Theory, 1976, 22(5):573~579
    [38] Jain Jaswant R and Jain Anil K. Displacement Measurement and Its Application in Interframe Image Coding. IEEE Transactions on Communications, 1981, 29(12):1799~1808
    [39] Ghanbari Mohammed. The Cross-Search Algorithm for Motion Estimation. IEEE Transactions on Communications, 1990, 38(7):950~953
    [40] Zhu Shan and Ma Kai-Kuang. A New Diamond Search Algorithm For Fast Block Matching Motion Estimation. In Proceedings of IEEE International Conference on Information, Communications and Signal Processing (ICICS’97), Beijing, China, 1997. 292~296
    [41] Lee Hwai-Suk, Jung Jik-Han and Park Dong-Jo. An Efficient Diamond Search with Large Kite Search Patterns for Fast Block Motion Estimation. In Proceedings of International Joint Conference on SICE- ICCAS, Busan, Korea, 2006. 3137~3141
    [42] Stephan Wenger. H.264/AVC Over IP. IEEE Transactions on Circuits and Systems for Video Technology, 2003, 13(7):645~656

© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号

地址:北京市海淀区学院路29号 邮编:100083

电话:办公室:(+86 10)66554848;文献借阅、咨询服务、科技查新:66554700