H.264空间分辨率转码研究
摘要
随着网络和多媒体通信技术的发展,在移动视频终端上提供视频服务的需求越来越强烈。移动终端具有一些共同的特点,包括显示分辨率低,芯片处理能力弱,同时受限于移动网络的带宽。MPEG-2是目前使用最广泛的视频编码标准之一,然而它压缩后的视频码率较高,并不适合于移动无线网络的视频需求。H.264是最新的视频编码标准,由于其出色的编码性能,将得到广泛的应用。本文针对移动视频终端的需求,研究MPEG-2视频和H.264视频之间的转换编码技术。
在简要介绍视频编码原理,特别是MPEG-2和H.264编码标准的基础上,对视频转换编码技术进行总结和归纳。论文的主要工作如下:
首先,提出了一种MPEG-2到H.264空间分辨率转码算法。它通过充分利用MPEG-2解码过程中得到的运动矢量和预测残差,可显著减小H.264编码过程中宏码编码模式确定和运动估计过程的计算复杂度,并得到最终的H.264视频流。PC平台上几种典型视频测试序列的仿真实验结果表明,本算法对视频质量的损失较小,有稳定的率失真性能,有利于实时转码的实现。
其次,考虑到H.264的编码过程具有很高的计算复杂度,对编码过程进行了优化,提出了一种快速的多参考帧运动估计算法。通过分析典型的视频序列选取参考帧的统计信息和相邻块所选择的参考帧,来调节当前编码块在每个参考帧中的运动搜索半径。该算法可以大幅度提高运动估计的速度,同时可以获得和参考软件相似的视频质量和比特率,有利于H.264编码器的实时应用。
With the development of network and multimedia communication technology, the need for providing video services on mobile devices is stronger and stronger. For these mobile terminals, they share some common characteristics, such as small display screen, low computational ability etc. Moreover, they are restricted by the bandwidth of mobile wireless network. MPEG-2 is one of the most widely used video coding standards. Because of its high bit rate after compression, it is not appropriate for application in wireless network. H.264 is the latest video standard. Because of its excellent performance, it will be widely used in near future. To meet the need of mobile video terminals, this thesis researches on the video transcoding between MPEG-2 and H.264.
After a brief introduction of video coding principles, especially the MPEG-2 and H.264 video standards, the techniques for video transcoding are summarized. The main work of this thesis is outlined as follows.
First, a spatial downscaling transcoding algorithm between MPEG-2 and H.264 is proposed. It full utilizes the motion vectors and predictive residues gathered in MPEG-2 decoding to greatly decrease the computational complexity of macroblock coding mode decision and motion vector re-estimation in H.264, and generates the H.264 bit streams. Experimental results on several video sequences demonstrate that it can significantly reduce the computational complexity of the transcoder with just slight sacrificing the rate-distortion performance, which does benefit to real time transcoding.
Second, a fast multiple reference frames selection algorithm is proposed to decrease the high computational complexity in H.264 encoder. It adjusts the search range of every reference frame according to the analysis of the probability of frequently occurred best reference frame and the best reference frames of neighboring blocks. The proposed approach is faster than the reference software, while keeping almost the same video quality and bit rate. It can do benefit to the real time application of H.264 codec.
在简要介绍视频编码原理,特别是MPEG-2和H.264编码标准的基础上,对视频转换编码技术进行总结和归纳。论文的主要工作如下:
首先,提出了一种MPEG-2到H.264空间分辨率转码算法。它通过充分利用MPEG-2解码过程中得到的运动矢量和预测残差,可显著减小H.264编码过程中宏码编码模式确定和运动估计过程的计算复杂度,并得到最终的H.264视频流。PC平台上几种典型视频测试序列的仿真实验结果表明,本算法对视频质量的损失较小,有稳定的率失真性能,有利于实时转码的实现。
其次,考虑到H.264的编码过程具有很高的计算复杂度,对编码过程进行了优化,提出了一种快速的多参考帧运动估计算法。通过分析典型的视频序列选取参考帧的统计信息和相邻块所选择的参考帧,来调节当前编码块在每个参考帧中的运动搜索半径。该算法可以大幅度提高运动估计的速度,同时可以获得和参考软件相似的视频质量和比特率,有利于H.264编码器的实时应用。
With the development of network and multimedia communication technology, the need for providing video services on mobile devices is stronger and stronger. For these mobile terminals, they share some common characteristics, such as small display screen, low computational ability etc. Moreover, they are restricted by the bandwidth of mobile wireless network. MPEG-2 is one of the most widely used video coding standards. Because of its high bit rate after compression, it is not appropriate for application in wireless network. H.264 is the latest video standard. Because of its excellent performance, it will be widely used in near future. To meet the need of mobile video terminals, this thesis researches on the video transcoding between MPEG-2 and H.264.
After a brief introduction of video coding principles, especially the MPEG-2 and H.264 video standards, the techniques for video transcoding are summarized. The main work of this thesis is outlined as follows.
First, a spatial downscaling transcoding algorithm between MPEG-2 and H.264 is proposed. It full utilizes the motion vectors and predictive residues gathered in MPEG-2 decoding to greatly decrease the computational complexity of macroblock coding mode decision and motion vector re-estimation in H.264, and generates the H.264 bit streams. Experimental results on several video sequences demonstrate that it can significantly reduce the computational complexity of the transcoder with just slight sacrificing the rate-distortion performance, which does benefit to real time transcoding.
Second, a fast multiple reference frames selection algorithm is proposed to decrease the high computational complexity in H.264 encoder. It adjusts the search range of every reference frame according to the analysis of the probability of frequently occurred best reference frame and the best reference frames of neighboring blocks. The proposed approach is faster than the reference software, while keeping almost the same video quality and bit rate. It can do benefit to the real time application of H.264 codec.
引文
[1] Wiegand T, Sullivan G J, Luthra A, et a1. Draft ITU-T Recommendation and Final Daft International Standard of Joint Video Specification (ITU-T Rec. H.264 I ISO/IEC 14496-10 AVC). In: Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG. Switzerland,2003,1-249
[2] 高文. 多媒体数据压缩技术. 北京:电子工业出版社,1994,1-6
[3] 毕厚杰 . 新一代视频压缩编码标准— H.264/AVC. 北京 :人民邮电出版社 , 2005,85-86
[4] ITU-T, ISO/IEC JTC1.Generic Coding of Moving Pictures and Associated Audio Information-Part 2: Video.In: ITU-T Recommendation H.262-ISO/IEC 13818-2 (MPEG-2). Switzerland, 1994, 1-202
[5] 钟玉琢,乔秉新,祁卫.运动图像及其伴音通用编码国际标准—MPEG-2. 北京:清华大学出版社,1999,1-374
[6] Ahmad I, Wei X H, Sun Y, et a1.Video transcoding: an overview of various techniques and research issues. IEEE Transactions on Multimedia, 2005, 7(5): 793-804
[7] Vetro A, Christopoulos C, Sun H F. Video transcoding architectures and techniques: an overview .IEEE signal processing magazine, 2003, 20(2): 18-29
[8] 褚晶辉,俞斯乐,鲁照华. 视频转换编码及其实现技术的研究.电子学报,2004, 32(10):1678-1683
[9] Bjork N, Christopoulos C. Transcoder architectures for video coding. IEEE Trans. Consumer Electron, 1998, 44(1):88-98
[10] Shen B, Sethi I K, Vasudev B.Adaptive Motion-Vector Resampling for Compressed Video Downscaling. IEEE Transactions on Circuits and Systems for Video Technology, 1999, 9(6): 929-936
[11] Hashemi M R, Winger L, panchanathan S.Compressed Domain Motion Vector Resampling for Downscaling of MPEG Video.In: International Conference On Image Processing. Kobe, Japan, 1999, 276-279
[12] Masaru Sugano, Yasuyuki Nakajima, Akio Yoneyama. An efficient transcoding from MPEG2 to MPEG4.In: International Conference on Image processing. Thessaloniki, Greece,2001,417-420
[13] Yin P, Vetro A, Liu B, et a1. Drift compensation for reduced spatial resolutiontranscoding. IEEE Trans. Circuits Syst. Video Technol., 2002, 12(11):1009-1020
[14] Chang S F, Messerschmidt D G. Manipulation and composting of MC-DCT compressed video. IEEE Journal Selected Area Communications, 1995, 13(1): 1-11
[15] Skodras A N, Christopoulos C. Down-sampling of compressed images in the DCT domain. In: Proc. European Conference on Signal Processing (EUSIPCO).Rhodes, Greece,1998,1713-1716
[16] Hwang Jenq-Neng, Wu Tzong-Der, Lin Chia-Wen.Dynamic frame-skipping in video transcoding.In: IEEE Second Workshop on Multimedia Signal Processing. Redondo Beach,CA, USA,1998,616-621
[17] Youn Jeongnam, Sun Ming-Ting. A fast motion vector composition method for temporal transcoding.In: IEEE International Symposium on Circuits and Systems. Florida, USA,1999, 243-246
[18] Chen Mei-Juan,Chu Ming-Chung, Pan Chih-Wei.Efficient motion-estimation algorithm for reduced frame-rate video transcoding.IEEE Trans. Circuits and Systems on Video Technology,2002,12(4):269-275
[19] Dogan S, Sadka A H, Kondoz A M. Efficient MPEG4/H.263 video transcoder for interoperability of heterogeneous multimedia networks. Electronics Letters,1999, 35(11):863-864
[20] Shanableh T, Ghanbari M.Heterogeneous video transcoding to lower spatio- temporal resolutions and different encoding formats. IEEE Trans. Multimedia, 2000,2(2):101-110
[21] Sun H, Kwok W, Zdepski J.Architectures for MPEG compressed bitstream scaling. IEEE Trans. Circuits System Video Technology,1996,6(2):191-199
[22] Choi B S, Chun K W, Xin J, et a1. An HDTV-to-SDTV spatial transcoder.IEEE Trans. on Circuits and Systems for Video Technology, 2002, 12(11): 998-1008
[23] Zhang Peng, Lu Yan, Huang Qing-Ming, et al. Mode mapping method for h.264/avc spatial downscaling transcoding. In: International Conference on Image Processing .Singapore, 2004, 2781-2784
[24] Zhou Z, Sun M T. Fast Macroblock Inter mode decision for H.264/MPEG-4 AVC. In:International Conference on Image Processing.Singapore, 2004, 789-792
[25] Zhou Z, Sun S, Lei S, et a1. Motion information and coding mode reuse for mpeg-2 to h.264 transcoding. In: IEEE International Symposium on Circuits and Systems. Kobe, Japan ,2005,1230-1233
[26] Lu X A, Tourapis A M, Yin P, et a1. Fast mode decision and motion estimationfor h.264 with a focus on mpeg2/h.264transcoding. In: IEEE International Symposium on Circuits and Systems. Kobe, Japan ,2005, 1246-1249
[27] Xin J, Vetro A, Sun H F. Converting DCT Coefficients to H.264/AVC transform coefficients.In: Proc IEEE PCM.Tokyo, Japan, 2004, 939-946
[28] Fernandez-Escribano, Cuenca G, Orozco-Barbosa P, et a1. Computational Complexity Reduction of Intra-frame Prediction in MPEG-2/H.264 Video Transcoders. In: IEEE Internatinal Conference on Multimedia and Expo. Amsterdam, The Netherlands, 2005,707-710
[29] 陈功,黄祥林,沈兰荪. 视频转码技术. 测控技术,2003,22(5):36-39
[30] Xin Jun, Lin Chia-Wen, Sun Ming-Ting. Digital Video Transcoding. Proceedings of the IEEE, 2005, 93(1):84-97
[31] 杜耀刚,蔡安妮. DCT 域视频转码技术综述.电子学报,2005,32(10):1644-1648
[32] Liang Y Q, Tan Y P. A New Content-Based Hybrid Video Transcoding Method. In: IEEE International Conference on Image Processing.Thessaloniki,Greece, 2001, 429-432
[33] MPEG Software Simulation Group (MSSG). Free MPEG software.http://www. mpeg.org/pub_ftp/mpeg/mssg/mpeg2vidcodec_v12.tar.gz, 1994-05-20
[34] Karsten Suehring. H.264/AVC Reference Software JM86. http://iphome.hhi.de/ suehring/tml/download/,2004-07-07
[35] Wiegand T, Sullivan G J, Bjntegaard G, et a1. Overview of the H.264/AVC video coding standard. IEEE Trans. Circuits Syst. Video Technol.,2003,13(7): 560-576
[36] Chang A, Au O C, Yeung Y M. A novel approach to fast multi-frame selection for H.264 video coding. In: International Symposium on Circuits and System. Thailand ,2003, 704 -707
[37] Shen Y, Wuang C, Yu L, et a1. Fast Multi-Frame Motion Estimation Algorithm in H.264.In: ICSP’04 Proceedings. China,2004,1187-1190
[38] Huang Y W, Hsieh B Y, Chien S Y, et a1.Analysis and Complexity Reduction of Multiple Reference Frames Motion Estimation in H.264/AVC. IEEE Transactions On Circuits and Systems for video technology,2006, 16(4):507-521
[39] Hsieh C H, Lu P C, Shyn J S, et a1. Motion Estimation Algorithm Using Interblock Correlation. Electronics Letters,1990,26(5): 276-277
[40] Sullivan G J, Wiegand T. Rate-distortion optimization for video compression. IEEE Signal Process. Mag., 1998,15(6): 74-90
[41] Wiegand T, Girod B. Lagrangian multiplier selection in hybrid video coder control.In: Proc.IEEE International Conference on Image Processing.Greece,2001, 542-545
[42] Li H J, Hsu C T, Chen M J. Fast Multiple Reference Frame Selection Method for Motion Estimation in JVT/H.264.In:IEEE Asia-Pacific Conference on Circuits and Systems.Taiwan,2004,605-608
[2] 高文. 多媒体数据压缩技术. 北京:电子工业出版社,1994,1-6
[3] 毕厚杰 . 新一代视频压缩编码标准— H.264/AVC. 北京 :人民邮电出版社 , 2005,85-86
[4] ITU-T, ISO/IEC JTC1.Generic Coding of Moving Pictures and Associated Audio Information-Part 2: Video.In: ITU-T Recommendation H.262-ISO/IEC 13818-2 (MPEG-2). Switzerland, 1994, 1-202
[5] 钟玉琢,乔秉新,祁卫.运动图像及其伴音通用编码国际标准—MPEG-2. 北京:清华大学出版社,1999,1-374
[6] Ahmad I, Wei X H, Sun Y, et a1.Video transcoding: an overview of various techniques and research issues. IEEE Transactions on Multimedia, 2005, 7(5): 793-804
[7] Vetro A, Christopoulos C, Sun H F. Video transcoding architectures and techniques: an overview .IEEE signal processing magazine, 2003, 20(2): 18-29
[8] 褚晶辉,俞斯乐,鲁照华. 视频转换编码及其实现技术的研究.电子学报,2004, 32(10):1678-1683
[9] Bjork N, Christopoulos C. Transcoder architectures for video coding. IEEE Trans. Consumer Electron, 1998, 44(1):88-98
[10] Shen B, Sethi I K, Vasudev B.Adaptive Motion-Vector Resampling for Compressed Video Downscaling. IEEE Transactions on Circuits and Systems for Video Technology, 1999, 9(6): 929-936
[11] Hashemi M R, Winger L, panchanathan S.Compressed Domain Motion Vector Resampling for Downscaling of MPEG Video.In: International Conference On Image Processing. Kobe, Japan, 1999, 276-279
[12] Masaru Sugano, Yasuyuki Nakajima, Akio Yoneyama. An efficient transcoding from MPEG2 to MPEG4.In: International Conference on Image processing. Thessaloniki, Greece,2001,417-420
[13] Yin P, Vetro A, Liu B, et a1. Drift compensation for reduced spatial resolutiontranscoding. IEEE Trans. Circuits Syst. Video Technol., 2002, 12(11):1009-1020
[14] Chang S F, Messerschmidt D G. Manipulation and composting of MC-DCT compressed video. IEEE Journal Selected Area Communications, 1995, 13(1): 1-11
[15] Skodras A N, Christopoulos C. Down-sampling of compressed images in the DCT domain. In: Proc. European Conference on Signal Processing (EUSIPCO).Rhodes, Greece,1998,1713-1716
[16] Hwang Jenq-Neng, Wu Tzong-Der, Lin Chia-Wen.Dynamic frame-skipping in video transcoding.In: IEEE Second Workshop on Multimedia Signal Processing. Redondo Beach,CA, USA,1998,616-621
[17] Youn Jeongnam, Sun Ming-Ting. A fast motion vector composition method for temporal transcoding.In: IEEE International Symposium on Circuits and Systems. Florida, USA,1999, 243-246
[18] Chen Mei-Juan,Chu Ming-Chung, Pan Chih-Wei.Efficient motion-estimation algorithm for reduced frame-rate video transcoding.IEEE Trans. Circuits and Systems on Video Technology,2002,12(4):269-275
[19] Dogan S, Sadka A H, Kondoz A M. Efficient MPEG4/H.263 video transcoder for interoperability of heterogeneous multimedia networks. Electronics Letters,1999, 35(11):863-864
[20] Shanableh T, Ghanbari M.Heterogeneous video transcoding to lower spatio- temporal resolutions and different encoding formats. IEEE Trans. Multimedia, 2000,2(2):101-110
[21] Sun H, Kwok W, Zdepski J.Architectures for MPEG compressed bitstream scaling. IEEE Trans. Circuits System Video Technology,1996,6(2):191-199
[22] Choi B S, Chun K W, Xin J, et a1. An HDTV-to-SDTV spatial transcoder.IEEE Trans. on Circuits and Systems for Video Technology, 2002, 12(11): 998-1008
[23] Zhang Peng, Lu Yan, Huang Qing-Ming, et al. Mode mapping method for h.264/avc spatial downscaling transcoding. In: International Conference on Image Processing .Singapore, 2004, 2781-2784
[24] Zhou Z, Sun M T. Fast Macroblock Inter mode decision for H.264/MPEG-4 AVC. In:International Conference on Image Processing.Singapore, 2004, 789-792
[25] Zhou Z, Sun S, Lei S, et a1. Motion information and coding mode reuse for mpeg-2 to h.264 transcoding. In: IEEE International Symposium on Circuits and Systems. Kobe, Japan ,2005,1230-1233
[26] Lu X A, Tourapis A M, Yin P, et a1. Fast mode decision and motion estimationfor h.264 with a focus on mpeg2/h.264transcoding. In: IEEE International Symposium on Circuits and Systems. Kobe, Japan ,2005, 1246-1249
[27] Xin J, Vetro A, Sun H F. Converting DCT Coefficients to H.264/AVC transform coefficients.In: Proc IEEE PCM.Tokyo, Japan, 2004, 939-946
[28] Fernandez-Escribano, Cuenca G, Orozco-Barbosa P, et a1. Computational Complexity Reduction of Intra-frame Prediction in MPEG-2/H.264 Video Transcoders. In: IEEE Internatinal Conference on Multimedia and Expo. Amsterdam, The Netherlands, 2005,707-710
[29] 陈功,黄祥林,沈兰荪. 视频转码技术. 测控技术,2003,22(5):36-39
[30] Xin Jun, Lin Chia-Wen, Sun Ming-Ting. Digital Video Transcoding. Proceedings of the IEEE, 2005, 93(1):84-97
[31] 杜耀刚,蔡安妮. DCT 域视频转码技术综述.电子学报,2005,32(10):1644-1648
[32] Liang Y Q, Tan Y P. A New Content-Based Hybrid Video Transcoding Method. In: IEEE International Conference on Image Processing.Thessaloniki,Greece, 2001, 429-432
[33] MPEG Software Simulation Group (MSSG). Free MPEG software.http://www. mpeg.org/pub_ftp/mpeg/mssg/mpeg2vidcodec_v12.tar.gz, 1994-05-20
[34] Karsten Suehring. H.264/AVC Reference Software JM86. http://iphome.hhi.de/ suehring/tml/download/,2004-07-07
[35] Wiegand T, Sullivan G J, Bjntegaard G, et a1. Overview of the H.264/AVC video coding standard. IEEE Trans. Circuits Syst. Video Technol.,2003,13(7): 560-576
[36] Chang A, Au O C, Yeung Y M. A novel approach to fast multi-frame selection for H.264 video coding. In: International Symposium on Circuits and System. Thailand ,2003, 704 -707
[37] Shen Y, Wuang C, Yu L, et a1. Fast Multi-Frame Motion Estimation Algorithm in H.264.In: ICSP’04 Proceedings. China,2004,1187-1190
[38] Huang Y W, Hsieh B Y, Chien S Y, et a1.Analysis and Complexity Reduction of Multiple Reference Frames Motion Estimation in H.264/AVC. IEEE Transactions On Circuits and Systems for video technology,2006, 16(4):507-521
[39] Hsieh C H, Lu P C, Shyn J S, et a1. Motion Estimation Algorithm Using Interblock Correlation. Electronics Letters,1990,26(5): 276-277
[40] Sullivan G J, Wiegand T. Rate-distortion optimization for video compression. IEEE Signal Process. Mag., 1998,15(6): 74-90
[41] Wiegand T, Girod B. Lagrangian multiplier selection in hybrid video coder control.In: Proc.IEEE International Conference on Image Processing.Greece,2001, 542-545
[42] Li H J, Hsu C T, Chen M J. Fast Multiple Reference Frame Selection Method for Motion Estimation in JVT/H.264.In:IEEE Asia-Pacific Conference on Circuits and Systems.Taiwan,2004,605-608