基于视觉的视频传输抗误码技术研究与系统实现
|
摘要
网络技术的迅速发展使多媒体通信的应用和研究受到前所未有的重视，但由于目前网络有限的带宽、Internet的分组丢失率、无线网络的高误码率、网络异构性以及网络延迟等多种因素的影响。使这一技术的发展面临着严峻的挑战。这里，本文针对无线网络有限的带宽，严重的误码现象和移动设备有限的计算与存储能力等问题，设计并实现了一个主要是面向无线应用，并综合了实时MPEG-4编码、传输和解码的实时多媒体通信系统。在该系统设计与实现过程中，获得了一些有特色的研究成果：
1、在有限的网络带宽和高误码率的信道环境下，如何利用人体视觉模型的特性来合理的使用网络带宽，减少因网络传输而发生的丢包或误码现象对视频重建图像质量的影响，是非常有意义的。对此，本文在对视频编解码技术以及当前抗误码技术研究的基础上，主要是针对无线局域网(WLAN)环境下的高误码率和严重的分组丢失问题，设计了一种基于视觉质量控制策略的综合抗误码方法(IERV)。该方法是在权衡考虑影响视频主观质量多种因素的基础上，并综合了传输、鲁棒性编码和解码端错误隐藏等技术的抗误码方法。在模拟实验中测试表明，采用基于视觉质量为控制策略的综合抗误码方法，在高误码的环境下，可使重建视频图像的PSNR值提高3～11分贝(db)。该方法已运用到已经申请软件著作权登记的“无线流媒体服务系统(W-Media V1.0)”中，收到了较好的效果。
2、设计了一种适合流式传输的解码器模型。该模型具有很好的扩展性，且非常适合用目前流行的多线程机制来实现，从而可在很大程度上提高编程和系统运行效率。
3、针对多媒体数据流分开存储、传输、接收与解码的特点，提出了一种与该解码器模型相适应的多媒体数据同步方法——基于全局时钟参考的同步法。该方法实现简单，且同步效果较好。
4、设计了一种可随机检测的视频水印算法，其与MPEG-4标准相结合。在纹理信息的Intra模式的DCT系数块中嵌入水印，并通过DC和AC系数预测方向来生成控制码，以控制水印密钥和水印信息的同步，从而有效地解决了视频水印检测的实时性和随机性问题。
With the advantage of network technology, the applications of multimedia communication have undergone enormous attention lately. However, the research of streaming media faces severe challenges for Limited Bandwidth, High bit error rate over wireless networks, Pocket Lost Rate in Internet, Heterogeneity and Delay of Networks. In this paper, we propose a real time multimedia communication system based on the wireless network and PDA. With the technology of foveation based coding, integration of error resilience for the MPEG-4 real time coding and wireless transmitting, it could improve video quality even under the limited network bandwidth, high bit error rate (BER) in wireless networks, weaker power and deficient storage for mobile terminal unit. Experiments prove that it can perform well in the WLAN and PDA. In the course of the design and implementation of the system, some impressive contributions are listed as follows.
1. It is very important to improve video quality by using the model of human vision system to decrease the effect of pocket lost or BER under the limited network bandwidth, high bit error rate (BER) in wireless networks. So a novel integrated error resilience method based on vision quality(IERV) under the high pocket lost rate and high bit error rate(BER) in wireless networks, which integrates the technology of robust coding and error concealment, is presented in this paper. With weighting the factors of video subjective quality, it integrates the technology of transmission, robust coding and error resilience of the decoder.Simulation results show that the objective and subjective video is obviously improved using proposed integrated error resilience technology with control of vision quality.
2.This paper designs a model of decoder for streaming media. For according to the popular multithread mechanism, it is easy to expand and improve the efficiency of development and decoding.
3.Moreover, This paper presents a method of multimedia-synthesized synchronization based on global clock. For multimedia stream is stored, transmitted, received and decoded individually, the method is easly realized.
4. This paper designs a scheme of random video watermark for MPEG-4, which embed the watermark in the DCT coefficient block of the intra mode of the texture information and generate the control code by the prediction directions of the DC and AC coefficient to synthesize the watermark and the secret key of watermark. It can efficiently detect the watermark in real time and in random.
1、在有限的网络带宽和高误码率的信道环境下，如何利用人体视觉模型的特性来合理的使用网络带宽，减少因网络传输而发生的丢包或误码现象对视频重建图像质量的影响，是非常有意义的。对此，本文在对视频编解码技术以及当前抗误码技术研究的基础上，主要是针对无线局域网(WLAN)环境下的高误码率和严重的分组丢失问题，设计了一种基于视觉质量控制策略的综合抗误码方法(IERV)。该方法是在权衡考虑影响视频主观质量多种因素的基础上，并综合了传输、鲁棒性编码和解码端错误隐藏等技术的抗误码方法。在模拟实验中测试表明，采用基于视觉质量为控制策略的综合抗误码方法，在高误码的环境下，可使重建视频图像的PSNR值提高3～11分贝(db)。该方法已运用到已经申请软件著作权登记的“无线流媒体服务系统(W-Media V1.0)”中，收到了较好的效果。
2、设计了一种适合流式传输的解码器模型。该模型具有很好的扩展性，且非常适合用目前流行的多线程机制来实现，从而可在很大程度上提高编程和系统运行效率。
3、针对多媒体数据流分开存储、传输、接收与解码的特点，提出了一种与该解码器模型相适应的多媒体数据同步方法——基于全局时钟参考的同步法。该方法实现简单，且同步效果较好。
4、设计了一种可随机检测的视频水印算法，其与MPEG-4标准相结合。在纹理信息的Intra模式的DCT系数块中嵌入水印，并通过DC和AC系数预测方向来生成控制码，以控制水印密钥和水印信息的同步，从而有效地解决了视频水印检测的实时性和随机性问题。
With the advantage of network technology, the applications of multimedia communication have undergone enormous attention lately. However, the research of streaming media faces severe challenges for Limited Bandwidth, High bit error rate over wireless networks, Pocket Lost Rate in Internet, Heterogeneity and Delay of Networks. In this paper, we propose a real time multimedia communication system based on the wireless network and PDA. With the technology of foveation based coding, integration of error resilience for the MPEG-4 real time coding and wireless transmitting, it could improve video quality even under the limited network bandwidth, high bit error rate (BER) in wireless networks, weaker power and deficient storage for mobile terminal unit. Experiments prove that it can perform well in the WLAN and PDA. In the course of the design and implementation of the system, some impressive contributions are listed as follows.
1. It is very important to improve video quality by using the model of human vision system to decrease the effect of pocket lost or BER under the limited network bandwidth, high bit error rate (BER) in wireless networks. So a novel integrated error resilience method based on vision quality(IERV) under the high pocket lost rate and high bit error rate(BER) in wireless networks, which integrates the technology of robust coding and error concealment, is presented in this paper. With weighting the factors of video subjective quality, it integrates the technology of transmission, robust coding and error resilience of the decoder.Simulation results show that the objective and subjective video is obviously improved using proposed integrated error resilience technology with control of vision quality.
2.This paper designs a model of decoder for streaming media. For according to the popular multithread mechanism, it is easy to expand and improve the efficiency of development and decoding.
3.Moreover, This paper presents a method of multimedia-synthesized synchronization based on global clock. For multimedia stream is stored, transmitted, received and decoded individually, the method is easly realized.
4. This paper designs a scheme of random video watermark for MPEG-4, which embed the watermark in the DCT coefficient block of the intra mode of the texture information and generate the control code by the prediction directions of the DC and AC coefficient to synthesize the watermark and the secret key of watermark. It can efficiently detect the watermark in real time and in random.
引文
[1] ISO/IEC 11172,Information technology:Coding of moving pictures and associated audio for digital storage media at up to about 1.5Mbit/s,Part 1 :Systems;Part 2:Video;Part 3 :Audio;Part 4:Conformance Testing, 1993.
[2] 钟玉琢、王琪、赵黎等，MPEG-2运动图像压缩国际标准及MPEG的新进展，北京：清华大学出版社，ISBN 7-302-05068-6／TP.2965，2002年。
[3] 钟玉琢、王琪、贺玉文，基于对象的多媒体数据压缩编码国际标准——MPEG-4及其校验模型(第1版)，北京：科学出版社，ISBN 7-03-008775-5／TP.1430，2000年，14-471。
[4] ITU-T (International Telecommunication Union - Telecommunication Standardisation Sector) Recommendation H.261,"Video codec for audiovisual services at p x 64 kbit/s", 1993.
[5] 沈兰荪、卓力、田栋等著，视频编码与低速率传输，第1版，北京：电子工业出版社，ISBN 7-5053-7111-8／TN.1480，2001年。
[6] J.Rosenberg, H.Schulzrinne, "An RTP Payload Format for Generic Forward Error Correction". IETF Draft. RFC 2733.1999.
[7] D. Hoffman etc, RTP Payload Format for MPEG1/MPEG2 Video, IETF RFC, 1998.
[8] Y. Kikuchi et al.: "RTP Payload Format for MPEG-4 Audio/Visual Streams", IETF RFC 3016, November 2000.
[9] K. W. Richardson, UMTS overview, Electronics and Communication Engineering Journal. June 2000, Page(s): 93-100.
[10] B. P. Crow, I. Widjaja, J. G. Kim, and P. T. Sakai, IEEE 802.11 wireless local area networks, IEEE Comm. Magazine, Sept. 1997. Page(s): 116-126.
[11] J. Haartsen and S. Mattisson, Bluetooth -a new low-power radio interface providing short-range connectivity, Proceedings of IEEE, vol. 88, Issue 10, Oct. 2000. Page(s): 1651-1661.
[12] H. Aghvami and B. Jafarian, A vision of UMTS/IMT-2000 evolution, Electronics and Communication Engineering Journal, June 2000, Page(s): 148-152.
[13] L. Robert, N. Pissinou, and S. Makki, Third generation wireless network: the integration of GSM and mobile IP, IEEE Wireless Communications and Networking Conference
2000 (WCNC 2000), vol. 3, 2000, Page(s): 1291-1296.
[14] Yao Wang, Stepban Wenger, Jingtao Wen.et,Error resilience Video Coding Techniques,Real-Time video Communication over Unreliable NetWorks,IEEE Signal Processing Magazine, July 2000. Page(s): 61-82.
[15] 易开祥，石教英，孙鑫，数字水印技术研究进展，中国图象图形学报，2001，Vol．6(A)，No．2：111-117。
[16] Thomas wiegand, Gary J, Sullivan et." Overview of the H.264/AVC video coding standard",IEEE Transactions on circuits and systems fo video technology, July 2003,Pages: 1-19.
[17] 章毓晋编著，图像处理与分析(上册)，第1版，北京：清华大学出版社，ISBN 7-302-03343-9／TN.100，1999年,72-93。
[18] Gregory J. Conklin, Gray S.Greenbaum,Karl O.Lillevold,Alan F.Lippman, and YuriyA.Reznik,"Video Coding for Streaming Media Delivery on the Internet", IEEE trans on Circuits and systems for video technology, vol: 11, no 3, June 2001, Page(s):269-281.
[19] W. Li, "Streaming video profile in MPEG-4", IEEE trans. Circuits and Systems for Video Technology, special issue on streaming video, vol 11, no 3, 2001,Page(s): 301-317.
[20] Dapeng Wu,Yiwei Thomas Hou,Wenwu Zhu,Ya-Qin Zhang and Jon M.Peha, " Streaming Video over the Internet:Approaches and Directions" , IEEE trans on Circuits and systems for video technology, vol 11, no 3, June 2001 Page(s):282-300.
[21] 邹潇湘，彭聪，李锦涛，基于置换和单向函数的水印验证方案，计算机研究与发展，2002年，第39卷(增刊)：61-66。
[22] Sang H.Kang and Avideh Zakhor.Packet Scheduling Algorithm for Wireless Video Streaming. Video and Image processing Lab.U.C.Berkeley IEEE,2001 infocom, Page(s): 1698-1704.
[23] 宋彬，常义林，基于主观质量控制的数字视频综合抗误码技术，电子学报，2001年，第29卷(第7期)：919-922。
[24] Weiping Li," Overview of Fine Granularity Scalability in MPEG-4 Video Standard" IEEE trans on Circuits and systems for video technology, vol 11, no 3, June 2001, Page(s):301-317.
[25] 陈名华，何芸，基于数字水印的错误检测和定位技术，清华大学学报，2002年，
第42卷(第7期)：937-940。
[26] W-M Lam, A Reibman.An error concelment algorithm for images subject to channel errors[j].IEEE Trns.Image Processing, 1995,Page(s): 533-542.
[27] Yao wang,Qn-fan Zhu. Error control and concealment for video communication:A review[J].Proceedings of the IEEE1, 1998,Vol 86.No. 5,Page(s):974-997.
[28] Dapeng Wu, Yiwei Thomas Hou, Ya-Qin Zhang. Transporting Real-time Video over the Internet: Challenges and Approaches. Proceedings of the IEEE, Y. 2000.Vol. 88, No. 12, Page(s): 1855-1875.
[29] D. Mandelbaum. "An Adaptive-Feedback Coding Scheme using Incremental Redundancy". IEEE Trans. Info. Theory, 1974,Vol. IT-20(No. 3). Page(s): 388-389.
[30] L. Rizzo, V. Visano. "A Reliable Multicast Data Distribution Protocol based on Software FEC Techniques ". Proc. 4th IEEE Workshop Arch. And Implementation of High Perf. Comm. Sys. (HPCS 97). 1997,Page(s): 23-25.
[31] W. Tan and A. Zakhor, "Error Control for Video Multicast using Hierarchical FEC," Intl Conf Image Processing (ICIP) 99, Vol. 1, Kobe, Japan, Oct 1999, Page(s): 401-405.
[32] 蔡皖东编著，多媒体通信技术，第1版，西安：西安电子科技大学出版社， ISBN-7-5606-0921-X／TN.0159，2000年,5～48。
[33] W. S. Geisler, and J. S. Perry, "A real-time foveated multiresolution system for low-bandwidth video Communication",Proc. SPIE, vol. 3299, Jul. 1998, Page(s): 294-305.
[34] H. Schulzrinne, A. Rao, R. Lanphier, "RTSP: Real Time Streaming Protocol", RFC 2326, April 1998.
[35] M. Handley. V. Jacobson "SDP: Session Description Protocol". RFC 2327. April 1998.
[36] G. Roth, R. Sjberg, G. Liebl, T. Stockhammer, V. Varsa, and M. Karczewicz, "Common Test Conditions for RTP/IP over 3GPP/3GPP2, - Amendments and Software," ITU-T SG16 Doc. VCEG-M37, Santa Barbara, CA, USA, Sept. 2001.
[37] 严权锋，邹北骥，黄晁等，无线实时多媒体通信系统，微电子学与计算机，2003年，第20卷(第8期)：20-26。
[38] 史红周，黄晁，李锦涛，一种基于mp4文件的视频流关键帧索引播放方法，微电子学与计算机，2002年，第12卷(第7期)：25-27。
[39] Feedback-based error control for mobile video transmission, Girod, B.; Farber, N.
Proceedings of the IEEE, Volume: 87 Issue: 10, Oct. 1999, Page(s): 1707-1723.
[40] 王曜，奥斯特曼，张亚勤著，视频信号处理与通信，第1版，北京：清华大学出版社，ISBN 7-302-06156-4／TP.3685，2003年,472～555。
[41] QuickTime Streaming specification in PDF http ://developer.Apple.com/techpubs/quicktime/qtdevdocs/PDF/QTStreaming.pdf 2003-01-09
[42] Dennis R. Ankrum, "Viewing distance at computer workstations," Workplace Ergonomics, Sept./Oct. 1996, Page(s): 10-12.
[43] Frank Hartung, Bernd Girod. Watermarking of uncompressed and compressed video. Signal Processing, Special issue on copyright protection and access control for multimedia services, May 1998, Vol. 66, No. 3, Page(s): 283-301.
[44] 邹潇湘，李锦涛，基于扩频的非对称数字水印，计算机科学，2002年，第29卷(第9期)：63-65。
[2] 钟玉琢、王琪、赵黎等，MPEG-2运动图像压缩国际标准及MPEG的新进展，北京：清华大学出版社，ISBN 7-302-05068-6／TP.2965，2002年。
[3] 钟玉琢、王琪、贺玉文，基于对象的多媒体数据压缩编码国际标准——MPEG-4及其校验模型(第1版)，北京：科学出版社，ISBN 7-03-008775-5／TP.1430，2000年，14-471。
[4] ITU-T (International Telecommunication Union - Telecommunication Standardisation Sector) Recommendation H.261,"Video codec for audiovisual services at p x 64 kbit/s", 1993.
[5] 沈兰荪、卓力、田栋等著，视频编码与低速率传输，第1版，北京：电子工业出版社，ISBN 7-5053-7111-8／TN.1480，2001年。
[6] J.Rosenberg, H.Schulzrinne, "An RTP Payload Format for Generic Forward Error Correction". IETF Draft. RFC 2733.1999.
[7] D. Hoffman etc, RTP Payload Format for MPEG1/MPEG2 Video, IETF RFC, 1998.
[8] Y. Kikuchi et al.: "RTP Payload Format for MPEG-4 Audio/Visual Streams", IETF RFC 3016, November 2000.
[9] K. W. Richardson, UMTS overview, Electronics and Communication Engineering Journal. June 2000, Page(s): 93-100.
[10] B. P. Crow, I. Widjaja, J. G. Kim, and P. T. Sakai, IEEE 802.11 wireless local area networks, IEEE Comm. Magazine, Sept. 1997. Page(s): 116-126.
[11] J. Haartsen and S. Mattisson, Bluetooth -a new low-power radio interface providing short-range connectivity, Proceedings of IEEE, vol. 88, Issue 10, Oct. 2000. Page(s): 1651-1661.
[12] H. Aghvami and B. Jafarian, A vision of UMTS/IMT-2000 evolution, Electronics and Communication Engineering Journal, June 2000, Page(s): 148-152.
[13] L. Robert, N. Pissinou, and S. Makki, Third generation wireless network: the integration of GSM and mobile IP, IEEE Wireless Communications and Networking Conference
2000 (WCNC 2000), vol. 3, 2000, Page(s): 1291-1296.
[14] Yao Wang, Stepban Wenger, Jingtao Wen.et,Error resilience Video Coding Techniques,Real-Time video Communication over Unreliable NetWorks,IEEE Signal Processing Magazine, July 2000. Page(s): 61-82.
[15] 易开祥，石教英，孙鑫，数字水印技术研究进展，中国图象图形学报，2001，Vol．6(A)，No．2：111-117。
[16] Thomas wiegand, Gary J, Sullivan et." Overview of the H.264/AVC video coding standard",IEEE Transactions on circuits and systems fo video technology, July 2003,Pages: 1-19.
[17] 章毓晋编著，图像处理与分析(上册)，第1版，北京：清华大学出版社，ISBN 7-302-03343-9／TN.100，1999年,72-93。
[18] Gregory J. Conklin, Gray S.Greenbaum,Karl O.Lillevold,Alan F.Lippman, and YuriyA.Reznik,"Video Coding for Streaming Media Delivery on the Internet", IEEE trans on Circuits and systems for video technology, vol: 11, no 3, June 2001, Page(s):269-281.
[19] W. Li, "Streaming video profile in MPEG-4", IEEE trans. Circuits and Systems for Video Technology, special issue on streaming video, vol 11, no 3, 2001,Page(s): 301-317.
[20] Dapeng Wu,Yiwei Thomas Hou,Wenwu Zhu,Ya-Qin Zhang and Jon M.Peha, " Streaming Video over the Internet:Approaches and Directions" , IEEE trans on Circuits and systems for video technology, vol 11, no 3, June 2001 Page(s):282-300.
[21] 邹潇湘，彭聪，李锦涛，基于置换和单向函数的水印验证方案，计算机研究与发展，2002年，第39卷(增刊)：61-66。
[22] Sang H.Kang and Avideh Zakhor.Packet Scheduling Algorithm for Wireless Video Streaming. Video and Image processing Lab.U.C.Berkeley IEEE,2001 infocom, Page(s): 1698-1704.
[23] 宋彬，常义林，基于主观质量控制的数字视频综合抗误码技术，电子学报，2001年，第29卷(第7期)：919-922。
[24] Weiping Li," Overview of Fine Granularity Scalability in MPEG-4 Video Standard" IEEE trans on Circuits and systems for video technology, vol 11, no 3, June 2001, Page(s):301-317.
[25] 陈名华，何芸，基于数字水印的错误检测和定位技术，清华大学学报，2002年，
第42卷(第7期)：937-940。
[26] W-M Lam, A Reibman.An error concelment algorithm for images subject to channel errors[j].IEEE Trns.Image Processing, 1995,Page(s): 533-542.
[27] Yao wang,Qn-fan Zhu. Error control and concealment for video communication:A review[J].Proceedings of the IEEE1, 1998,Vol 86.No. 5,Page(s):974-997.
[28] Dapeng Wu, Yiwei Thomas Hou, Ya-Qin Zhang. Transporting Real-time Video over the Internet: Challenges and Approaches. Proceedings of the IEEE, Y. 2000.Vol. 88, No. 12, Page(s): 1855-1875.
[29] D. Mandelbaum. "An Adaptive-Feedback Coding Scheme using Incremental Redundancy". IEEE Trans. Info. Theory, 1974,Vol. IT-20(No. 3). Page(s): 388-389.
[30] L. Rizzo, V. Visano. "A Reliable Multicast Data Distribution Protocol based on Software FEC Techniques ". Proc. 4th IEEE Workshop Arch. And Implementation of High Perf. Comm. Sys. (HPCS 97). 1997,Page(s): 23-25.
[31] W. Tan and A. Zakhor, "Error Control for Video Multicast using Hierarchical FEC," Intl Conf Image Processing (ICIP) 99, Vol. 1, Kobe, Japan, Oct 1999, Page(s): 401-405.
[32] 蔡皖东编著，多媒体通信技术，第1版，西安：西安电子科技大学出版社， ISBN-7-5606-0921-X／TN.0159，2000年,5～48。
[33] W. S. Geisler, and J. S. Perry, "A real-time foveated multiresolution system for low-bandwidth video Communication",Proc. SPIE, vol. 3299, Jul. 1998, Page(s): 294-305.
[34] H. Schulzrinne, A. Rao, R. Lanphier, "RTSP: Real Time Streaming Protocol", RFC 2326, April 1998.
[35] M. Handley. V. Jacobson "SDP: Session Description Protocol". RFC 2327. April 1998.
[36] G. Roth, R. Sjberg, G. Liebl, T. Stockhammer, V. Varsa, and M. Karczewicz, "Common Test Conditions for RTP/IP over 3GPP/3GPP2, - Amendments and Software," ITU-T SG16 Doc. VCEG-M37, Santa Barbara, CA, USA, Sept. 2001.
[37] 严权锋，邹北骥，黄晁等，无线实时多媒体通信系统，微电子学与计算机，2003年，第20卷(第8期)：20-26。
[38] 史红周，黄晁，李锦涛，一种基于mp4文件的视频流关键帧索引播放方法，微电子学与计算机，2002年，第12卷(第7期)：25-27。
[39] Feedback-based error control for mobile video transmission, Girod, B.; Farber, N.
Proceedings of the IEEE, Volume: 87 Issue: 10, Oct. 1999, Page(s): 1707-1723.
[40] 王曜，奥斯特曼，张亚勤著，视频信号处理与通信，第1版，北京：清华大学出版社，ISBN 7-302-06156-4／TP.3685，2003年,472～555。
[41] QuickTime Streaming specification in PDF http ://developer.Apple.com/techpubs/quicktime/qtdevdocs/PDF/QTStreaming.pdf 2003-01-09
[42] Dennis R. Ankrum, "Viewing distance at computer workstations," Workplace Ergonomics, Sept./Oct. 1996, Page(s): 10-12.
[43] Frank Hartung, Bernd Girod. Watermarking of uncompressed and compressed video. Signal Processing, Special issue on copyright protection and access control for multimedia services, May 1998, Vol. 66, No. 3, Page(s): 283-301.
[44] 邹潇湘，李锦涛，基于扩频的非对称数字水印，计算机科学，2002年，第29卷(第9期)：63-65。