实时视频编码传输中H.264码率控制的研究与实现
|
摘要
在最近数十年的时间里,视频压缩技术和传输技术的长足发展使得视频通信成为了现实。作为ISO/IEC和ITU两大国际化标准组织联手制定的视频压缩标准,H.264以其优异的压缩性能和良好的网络亲和性得到了众多青睐,现在市场上已经出现了采用H.264编码的可视电话,足以说明H.264的重要价值。随着无线传输技术的成熟和进步,实时视频无线传输成为研究热点,这也催促着保持输出码率,使视频流适应信道带宽的视频码率控制技术不断改进和发展。同样是实时视频传输,在不同的应用中有着不同的侧重点。在生活娱乐中,人们对视频系统的要求往往是高清晰度的画面和逼真的视觉效果,而在工业和军事应用中,往往更为重要的却是毫厘不差的实时性和系统连续运转时的稳定性。本设计源于与科研单位的合作项目,研制的正是一个对实时性要求比较苛刻的视频传输系统,并需要为之设计适合的码率控制方案。
论文首先研究了H.264视频压缩标准,参考了大量相关资料,设计了基于DSP的实时视频编码系统,具体讨论了DSP软件的设计过程,包括驱动编写,代码移植和优化。然后分析了H.264的码流结构,讨论了码流在无线信道中传输的健壮性问题,为本设计选用了参数集固化的抗误码方法。
码率控制算法的研究与实现是论文的重点,论文研究了多种已有的码率控制方案,总结出常见的几种码率模型并分析了它们各自的优缺点。H.264编码由于采用了率失真优化技术,导致了其编码过程中的蛋鸡悖论,所以必须对传统码率控制方法进行重大修改,于是产生了H.264的众多码率控制提案。本文对这些提案进行了详尽的解读和深入的分析,讨论了前期提案对蛋鸡悖论的解决方法,以及后续提案对前期经典提案的改进。在此基础上,结合现阶段的研究成果,对码率控制的改进思想及相应方法进行了讨论。针对本设计的应用实际,本文的码率控制方案采用JVT-G012提案的码率控制框架,设计了低复杂度的宏块层码率控制策略,提出了一种结合运动矢量的时空域MAD预测法,充分考虑了图像的时空相关性,用于改进图像复杂度预测模型,将线性回归的计算复杂度降低为加权平均的复杂度。对率失真模型采用线性拟合代替二次拟合,极大地降低了运算复杂度。充分利用已编码单元信息对当前量化参数进行调整,有效控制码率。帧层缓冲区级别设置按实际缓冲区充盈度刷新,提高了帧层比特分配的合理性。实验证明,本文算法能有效的平滑峰值信噪比波动,并且算法复杂度低。
In the past few decades, video communication becomes true since the rapid development of video compression and transmission technology. As a standard made up by ITU and ISO/IEC, H.264 gets high performance in compression and works excellently in transmission. Video phone based on H.264 has been selling in the market, which illustrates the importance and extensive use of H.264. With the development of wireless transmission technology, the research of real-time video transmission becomes a hotspot, which pushes the improvement of rate control technology. Rate control is a technology which could subject the bit rate to a constraint value to fit the channel bandwidth. In Entertainment, people’s requirements for the video system are often high-definition and lifelike visual effects. While in the industrial and military applications, real-time performance and stability are more important, often. This design originates from a co-operation project with some scientific research unit, a real-time video compression and transmission system which has high requirement in real-time performance. A rate control schme is needed to be designed.
First of all, the paper presents a detailed analysis of key algorithms in H.264 principle by researching a lot of literature, and discusses the design of the video compression system based on DSP, including the bottom-chip peripherals driver programming, transplantation of T264 source code and the DSP realization and optimizing. Then, the bit stream of H.264 is analyzed, and the robustness via wireless transmission is also discussed. Based on the discussion, this design used two kinds of error resilience method, fixed parameters and redundant slice, for the system.
The emphasis of the paper is the research and realization of the rate control algorithm. After researching a lot of literature, several rate control models are analyzed, both the advantage and disadvantage.Compared with the former video coding standard, H.264 brought in the theory of rate distortion optimization, which lead to The Chicken and Egg Dilemma. So the conventional rate control schme need to be modified. The paper shows how the dilemma was solved in H.264 rate control scheme, analyzes the development and shortcomings in different schemes and then discusses several classical improved methods. The paper uses the framework of H.264 rate control scheme JVT-G012, and proposed a low complexity rate control algorithm in MB-layer. Making use of temporal and spatial correlation among frame, a temporal-spatial method to predict the MAD combining with motion vector is proposed.Using linear modes instead of secondary model reduces the computational complexity much. Using the information of coded MB to adjust the current MB’s quantization parameter makes the rate control more effectively.In bit allocation of frame level, by amending the conventional regulation of target buffer level, the bit allocation becomes more reasonable, and the standard deviation of PSNR is significantly improved.
论文首先研究了H.264视频压缩标准,参考了大量相关资料,设计了基于DSP的实时视频编码系统,具体讨论了DSP软件的设计过程,包括驱动编写,代码移植和优化。然后分析了H.264的码流结构,讨论了码流在无线信道中传输的健壮性问题,为本设计选用了参数集固化的抗误码方法。
码率控制算法的研究与实现是论文的重点,论文研究了多种已有的码率控制方案,总结出常见的几种码率模型并分析了它们各自的优缺点。H.264编码由于采用了率失真优化技术,导致了其编码过程中的蛋鸡悖论,所以必须对传统码率控制方法进行重大修改,于是产生了H.264的众多码率控制提案。本文对这些提案进行了详尽的解读和深入的分析,讨论了前期提案对蛋鸡悖论的解决方法,以及后续提案对前期经典提案的改进。在此基础上,结合现阶段的研究成果,对码率控制的改进思想及相应方法进行了讨论。针对本设计的应用实际,本文的码率控制方案采用JVT-G012提案的码率控制框架,设计了低复杂度的宏块层码率控制策略,提出了一种结合运动矢量的时空域MAD预测法,充分考虑了图像的时空相关性,用于改进图像复杂度预测模型,将线性回归的计算复杂度降低为加权平均的复杂度。对率失真模型采用线性拟合代替二次拟合,极大地降低了运算复杂度。充分利用已编码单元信息对当前量化参数进行调整,有效控制码率。帧层缓冲区级别设置按实际缓冲区充盈度刷新,提高了帧层比特分配的合理性。实验证明,本文算法能有效的平滑峰值信噪比波动,并且算法复杂度低。
In the past few decades, video communication becomes true since the rapid development of video compression and transmission technology. As a standard made up by ITU and ISO/IEC, H.264 gets high performance in compression and works excellently in transmission. Video phone based on H.264 has been selling in the market, which illustrates the importance and extensive use of H.264. With the development of wireless transmission technology, the research of real-time video transmission becomes a hotspot, which pushes the improvement of rate control technology. Rate control is a technology which could subject the bit rate to a constraint value to fit the channel bandwidth. In Entertainment, people’s requirements for the video system are often high-definition and lifelike visual effects. While in the industrial and military applications, real-time performance and stability are more important, often. This design originates from a co-operation project with some scientific research unit, a real-time video compression and transmission system which has high requirement in real-time performance. A rate control schme is needed to be designed.
First of all, the paper presents a detailed analysis of key algorithms in H.264 principle by researching a lot of literature, and discusses the design of the video compression system based on DSP, including the bottom-chip peripherals driver programming, transplantation of T264 source code and the DSP realization and optimizing. Then, the bit stream of H.264 is analyzed, and the robustness via wireless transmission is also discussed. Based on the discussion, this design used two kinds of error resilience method, fixed parameters and redundant slice, for the system.
The emphasis of the paper is the research and realization of the rate control algorithm. After researching a lot of literature, several rate control models are analyzed, both the advantage and disadvantage.Compared with the former video coding standard, H.264 brought in the theory of rate distortion optimization, which lead to The Chicken and Egg Dilemma. So the conventional rate control schme need to be modified. The paper shows how the dilemma was solved in H.264 rate control scheme, analyzes the development and shortcomings in different schemes and then discusses several classical improved methods. The paper uses the framework of H.264 rate control scheme JVT-G012, and proposed a low complexity rate control algorithm in MB-layer. Making use of temporal and spatial correlation among frame, a temporal-spatial method to predict the MAD combining with motion vector is proposed.Using linear modes instead of secondary model reduces the computational complexity much. Using the information of coded MB to adjust the current MB’s quantization parameter makes the rate control more effectively.In bit allocation of frame level, by amending the conventional regulation of target buffer level, the bit allocation becomes more reasonable, and the standard deviation of PSNR is significantly improved.
引文
[1]阮秋琦,阮宇智等译.数字图像处理[M].北京:电子工业出版社.第二版,2003.3.
[2] Rob Koenen.Overview of the MPEG-4 Standard[J].ISO/IEC JTC1/SC29/WG11 N1730 Stockholm,July 1997.
[3] ISO/IEC.Information Technology-Coding of Audio-Visual Objects: Visual[S].ISO/IEC 14496-2 Committee Draft of 15.May,1998.
[4] H.264/MPEG-4 Part 10 White Paper[EB/OL].www.vcodex.com,2004.7.
[5] ITU-T H.264建议书[S].国际电信联盟,2005.3.
[6] Description of Ref.Model8(RM8)[S]. CCITT SGXV, June.1989.
[7] Test Model5[S]. Draft, ISO-IEC/JTCI/SC29 AVGWGII, Apr.1993.
[8] Text of ISO/IEC 14496-2 MPEG-4 Video VM-Version 8[S]. Coding of Moving Pictures and Associated Audio MPEG 97/W1796, Stochholm, Sweden, July. 1997.
[9] Video Codec Test Model, ITU-T/SG15, TMN8[S], Portland, June. 1997.
[10] S. Ma, W. Gao. Proposed Draft Description of Rate Control on JVT Standard, JVT-F086[S]. Joint Video Team(JVT) of ISO/IEC MPEG & ITU-T VCEG 6th Meeting. Awaji, 2002:1~5.
[11]马思伟,高文,袁禄军,吕岩.一种面向H.264/AVC的码率控制算法[J].电子学报, 2004(12):2024~2027.
[12] Siwei Ma, Wen Gao, Yan Lu. Rate Control on JVT Standard, JVT-D030[S]. Joint Video Team(JVT) of ISO/IEC MPEG & ITU-T VCEG 4th meeting: Klagenfurt, Austria, 2002.
[13] Z.G. Li, F.Pan, K.P. Lim, et al. Adaptive Basic Unit Layer Rate Control for JVT, JVT-G012[S]. Joint Video Team(JVT) of ISO/IEC MPEG & ITU-T VCEG 7th meeting. Pattaya, Thailand, 2003:1~6.
[14] Institude of Computing Technology, Chinese Academy of Sciences, Institude for InfoComm Research, Singapore. Proposed Draft of Adaptive Rate Control, JVT-H017[S]. Joint Video Team(JVT) of ISO/IEC MPEG & ITU-T VCEG 8th meeting. Geneva, 2003:1~15.
[15] Yuan Wu, Lin Shouxun, Zhang Yongdong. Optimum Bit Allocation and Rate Control for H.264/AVC, JVT-O016[S]. Joint Video Team(JVT) of ISO/IEC MPEG & ITU-T VCEG 15th meeting. Busan, KR, 2005.4.
[16] Athanasios Leontaris, Alexis Michael Tourapis. Rate Control Reorganization in the Joint Model (JM) reference software, JVT-W042[S]. Joint Video Team(JVT) of ISO/IEC MPEG & ITU-T VCEG 23rd meeting. San Jose, California, USA, 2007.4.
[17] Iain E.G. Richardson. H.264 and MPEG-4 Video Compression[M]. The Robert GordonUniversity, Aberdeen, UK, 2004.
[18] Xilinx公司Wilson C. Chung.在FPGA上实现H.264/AVC视频编码标准[J].通信与计算机,2005.8.
[19] TMS320DM642 Video/Imaging Fixed-Point Digital Signal Processor[S].Texas instruments incorporated,2007.1.
[20] ITU-R.RECOMMENDATION ITU-R BT.656-4[S].ITU Radiocommunication Assembly,1998.
[21] Code Composer Studio Getting Started Guide (spru509c)[S].Texas Instruments Incorporated,2006.
[22] Texas Instruments Incorporated,田黎育等译.TMS320C6000系列DSP编程工具与指南[M].北京:清华大学出版社,2006.9.
[23] Rafael C.Gonzalez.数字图像处理(第二版)[M].北京:电子工业出版社,2003.7.
[24] TMS320C6000 Optimizing Compiler User's Guide(spru187k)[S] . Texas Instruments incorporated.2005.
[25] Wang Ning, He Yun. A new bitrate control strategy for H.264[J]. IEEE Transactions on Signal Processing, 2003, 15(12):1370-1374.
[26] LEEJ, DICKINSON B W. Temporally adaptive motion interpolation exploting temporal masking in visual perception[J]. IEEE Transactions on Image Processing, 1994,3(5):513-526.
[27] T. M. Cover, J. A. Thomas. Elements of information Theory[M]. New York, wiley, 1991.
[28] W. Ding, B. Liu. Rate Control of MPEG Video Coding and Recording by Rate-Quantization Modeling[J]. IEEE Trans. Circuits and Syst. for Video Technology, Feb. 1996, 6(1):12-20.
[29] K.R. Castleman, Digital image processing[M]. Copyright 1996 by Prentice-Hall, Inc.
[30] Tihao Chiang, Yaqin Zhang. A New Rate Control Scheme Using Quadratic Rate Distortion Model[J]. IEEE Transactions on Circuits and Systems for Video Technology.1997.2:246-250.
[31] Z.H.He, S.K.Mitra. A Linear Source Model and a Unified Rate Control Algorithm for DCT Video Coding[J]. IEEE Trans. Circuits systs. For Video Technology. Nov.2002,vol.12(11):970-982.
[32] Sullivan G J, Wiegand T. Rate distortion optimization for video compression[J]. IEEE Signal Processing Mag, vol.15, 1998,11:74-90.
[33] T. Chiang, Y.Q.Zhang. A New Rate Control Scheme using Quadratic Rate Distortion Model[J]. IEEE Trans. Circuits Syst. for Video Technology, Feb.1999.9:172-185.
[34]周树民,李锦涛,黄晁.低比特率视频编码中一种有效的码率分配算法[J].计算机辅助设计与图形学学报, 2005.11:2558-2564.
[35] Jiang M Q, Yi X Q and Ling N. Improved frame-layer rate control for H.264 using MADratio[J]. Circuits and Systems ISCAS apos, 2004,23(5):813-816.
[36]李晓辉,施芝元,张红伟.基于DCT系数分布特性的视频编码码率控制策略[J].厦门大学学报(自然科学版), 2001.6:1222-1227.
[37] Altunbasak Y, Kamaci N. An analysis of the DCT coefficient distribution with the H.264[J].IEEE International Conference. 2004.17(5):177-180.
[38] Moscheni F, Dufaux F, Nicolas H. Entropy criterion for optimal bit allocation between motion and prediction error information[J]. Proceedings of the SPIE Visual Commun and Image Proc. 1993:1548-1557.
[39] Wang Z, Bovik A C, Sheikh H R, Simoncelli E P. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004.13(4):600-612.
[40] Wang Z, Lu L, Bovik A C. Video quality assessment using structural distortion measurement[J]. Proceedings of the IEEE International Conference on Image Processing. Rochester, 2002:65-68.
[2] Rob Koenen.Overview of the MPEG-4 Standard[J].ISO/IEC JTC1/SC29/WG11 N1730 Stockholm,July 1997.
[3] ISO/IEC.Information Technology-Coding of Audio-Visual Objects: Visual[S].ISO/IEC 14496-2 Committee Draft of 15.May,1998.
[4] H.264/MPEG-4 Part 10 White Paper[EB/OL].www.vcodex.com,2004.7.
[5] ITU-T H.264建议书[S].国际电信联盟,2005.3.
[6] Description of Ref.Model8(RM8)[S]. CCITT SGXV, June.1989.
[7] Test Model5[S]. Draft, ISO-IEC/JTCI/SC29 AVGWGII, Apr.1993.
[8] Text of ISO/IEC 14496-2 MPEG-4 Video VM-Version 8[S]. Coding of Moving Pictures and Associated Audio MPEG 97/W1796, Stochholm, Sweden, July. 1997.
[9] Video Codec Test Model, ITU-T/SG15, TMN8[S], Portland, June. 1997.
[10] S. Ma, W. Gao. Proposed Draft Description of Rate Control on JVT Standard, JVT-F086[S]. Joint Video Team(JVT) of ISO/IEC MPEG & ITU-T VCEG 6th Meeting. Awaji, 2002:1~5.
[11]马思伟,高文,袁禄军,吕岩.一种面向H.264/AVC的码率控制算法[J].电子学报, 2004(12):2024~2027.
[12] Siwei Ma, Wen Gao, Yan Lu. Rate Control on JVT Standard, JVT-D030[S]. Joint Video Team(JVT) of ISO/IEC MPEG & ITU-T VCEG 4th meeting: Klagenfurt, Austria, 2002.
[13] Z.G. Li, F.Pan, K.P. Lim, et al. Adaptive Basic Unit Layer Rate Control for JVT, JVT-G012[S]. Joint Video Team(JVT) of ISO/IEC MPEG & ITU-T VCEG 7th meeting. Pattaya, Thailand, 2003:1~6.
[14] Institude of Computing Technology, Chinese Academy of Sciences, Institude for InfoComm Research, Singapore. Proposed Draft of Adaptive Rate Control, JVT-H017[S]. Joint Video Team(JVT) of ISO/IEC MPEG & ITU-T VCEG 8th meeting. Geneva, 2003:1~15.
[15] Yuan Wu, Lin Shouxun, Zhang Yongdong. Optimum Bit Allocation and Rate Control for H.264/AVC, JVT-O016[S]. Joint Video Team(JVT) of ISO/IEC MPEG & ITU-T VCEG 15th meeting. Busan, KR, 2005.4.
[16] Athanasios Leontaris, Alexis Michael Tourapis. Rate Control Reorganization in the Joint Model (JM) reference software, JVT-W042[S]. Joint Video Team(JVT) of ISO/IEC MPEG & ITU-T VCEG 23rd meeting. San Jose, California, USA, 2007.4.
[17] Iain E.G. Richardson. H.264 and MPEG-4 Video Compression[M]. The Robert GordonUniversity, Aberdeen, UK, 2004.
[18] Xilinx公司Wilson C. Chung.在FPGA上实现H.264/AVC视频编码标准[J].通信与计算机,2005.8.
[19] TMS320DM642 Video/Imaging Fixed-Point Digital Signal Processor[S].Texas instruments incorporated,2007.1.
[20] ITU-R.RECOMMENDATION ITU-R BT.656-4[S].ITU Radiocommunication Assembly,1998.
[21] Code Composer Studio Getting Started Guide (spru509c)[S].Texas Instruments Incorporated,2006.
[22] Texas Instruments Incorporated,田黎育等译.TMS320C6000系列DSP编程工具与指南[M].北京:清华大学出版社,2006.9.
[23] Rafael C.Gonzalez.数字图像处理(第二版)[M].北京:电子工业出版社,2003.7.
[24] TMS320C6000 Optimizing Compiler User's Guide(spru187k)[S] . Texas Instruments incorporated.2005.
[25] Wang Ning, He Yun. A new bitrate control strategy for H.264[J]. IEEE Transactions on Signal Processing, 2003, 15(12):1370-1374.
[26] LEEJ, DICKINSON B W. Temporally adaptive motion interpolation exploting temporal masking in visual perception[J]. IEEE Transactions on Image Processing, 1994,3(5):513-526.
[27] T. M. Cover, J. A. Thomas. Elements of information Theory[M]. New York, wiley, 1991.
[28] W. Ding, B. Liu. Rate Control of MPEG Video Coding and Recording by Rate-Quantization Modeling[J]. IEEE Trans. Circuits and Syst. for Video Technology, Feb. 1996, 6(1):12-20.
[29] K.R. Castleman, Digital image processing[M]. Copyright 1996 by Prentice-Hall, Inc.
[30] Tihao Chiang, Yaqin Zhang. A New Rate Control Scheme Using Quadratic Rate Distortion Model[J]. IEEE Transactions on Circuits and Systems for Video Technology.1997.2:246-250.
[31] Z.H.He, S.K.Mitra. A Linear Source Model and a Unified Rate Control Algorithm for DCT Video Coding[J]. IEEE Trans. Circuits systs. For Video Technology. Nov.2002,vol.12(11):970-982.
[32] Sullivan G J, Wiegand T. Rate distortion optimization for video compression[J]. IEEE Signal Processing Mag, vol.15, 1998,11:74-90.
[33] T. Chiang, Y.Q.Zhang. A New Rate Control Scheme using Quadratic Rate Distortion Model[J]. IEEE Trans. Circuits Syst. for Video Technology, Feb.1999.9:172-185.
[34]周树民,李锦涛,黄晁.低比特率视频编码中一种有效的码率分配算法[J].计算机辅助设计与图形学学报, 2005.11:2558-2564.
[35] Jiang M Q, Yi X Q and Ling N. Improved frame-layer rate control for H.264 using MADratio[J]. Circuits and Systems ISCAS apos, 2004,23(5):813-816.
[36]李晓辉,施芝元,张红伟.基于DCT系数分布特性的视频编码码率控制策略[J].厦门大学学报(自然科学版), 2001.6:1222-1227.
[37] Altunbasak Y, Kamaci N. An analysis of the DCT coefficient distribution with the H.264[J].IEEE International Conference. 2004.17(5):177-180.
[38] Moscheni F, Dufaux F, Nicolas H. Entropy criterion for optimal bit allocation between motion and prediction error information[J]. Proceedings of the SPIE Visual Commun and Image Proc. 1993:1548-1557.
[39] Wang Z, Bovik A C, Sheikh H R, Simoncelli E P. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004.13(4):600-612.
[40] Wang Z, Lu L, Bovik A C. Video quality assessment using structural distortion measurement[J]. Proceedings of the IEEE International Conference on Image Processing. Rochester, 2002:65-68.