开放体系下可扩展流媒体服务器系统研究
|
摘要
随着互连网的飞速发展,IPTV、视频点播流媒体应用发展迅速,迫切需要有较好的服务能力的媒体服务系统来支撑业务的开展。当前Real、微软和苹果公司都有发布各自宽带流媒体的解决方案,但这些方案往往需要高成本来支持。针对这种情况,本文通过对现有流媒体技术和产品的了解和分析,提出了开放体系下的可扩展媒体服务系统方案,采用现有的一些开源项目,来提高流媒体服务器的性能和可扩展能力。
本论文的主要研究内容有:
①介绍了现有的流媒体技术和集群技术,结合流媒体应用发展趋势分析了对流媒体服务器本身以及流媒体服务系统的需求。
②在详细分析LVS系统和Darwin流媒体服务器这两个开源项目的基础上,提出了可扩展的流媒体服务器体系结构,并介绍了该系统的工作原理和数据流程。
③在可扩展的流媒体服务器体系结构基础上,对分布式调度机制进行研究,并对负载均衡模块、接入许可控制机制、服务器调度算法进行了改进,提出了基于用户级别的接入许可控制机制和基于容量规划的接入控制优化策略,并实现了动态调度反馈机制。
④针对节点流媒体服务器在系统中的具体功能作用,对服务器的数据存储机制和流媒体服务策略等方面进行了进一步的研究,提出了基于节目流行度的数据存储策略和基于门限的流媒体服务器策略,来提高系统对媒体服务用户的支持能力。
⑤对系统的并发用户数和系统流量输出两个方面进行了测试,本文提出的可扩展流媒体服务器系统体系结构的确扩展了单台流媒体服务器的服务能力,而且具有较好的负载均衡性能。
本文提出的开放体系下可扩展流媒体服务器系统可以更好地发挥流媒体服务器系统中软件和硬件的综合服务能力,是一种解决小规模媒体服务应用的较好方案。该结构搭建比较简单,也不需要较高费用来支持,有较好的性价比,同时具有良好的可扩展性和节点透明性。
Along with the rapid evolution of the Internet, streaming media service such as IPTV and video-on-demand becomes more and more popular, so we need a high performance and flexible streaming server and system to support these services. The Real、Microsoft and Apple companies programs require high-cost support. According to the understanding and analysis of streaming media technology and product, we present a scalable streaming server system to expand media service based on open-frame. By using some of the existing open source projects, we can improve the streaming media server performance and scalability.
The main content is as follows:
①This paper at first introduces the streaming media technology development and the cluster technology. Then it proposes the requirement of streaming media server itself and streaming media service system.
②After detailed analysis of LVS and Darwin server systems, we design a scalable streaming media server architecture based on the two open source projects. Then it introduces its working principle and data flowing.
③Based on the expansion of the streaming media server architecture, the paper The paper proposes the permission control strategy based on user priority and the permission algorithm based on the capacity control and the dynamic feedback scheduling mechanism.
④According to the function of streaming media sever, the paper improves the data storage mechanism and streaming media services strategy. It proposes the storage mechanism based on program popularity and streaming media services strategy based on threshold.
⑤After testing the system concurrent number and the server export, the result indicated the system scalability and load balancing.
The scalable streaming server system improves the services support to users and makes the hardware and software of streaming media server system cooperates much better. It should be a good solution to small-scale media services program.
本论文的主要研究内容有:
①介绍了现有的流媒体技术和集群技术,结合流媒体应用发展趋势分析了对流媒体服务器本身以及流媒体服务系统的需求。
②在详细分析LVS系统和Darwin流媒体服务器这两个开源项目的基础上,提出了可扩展的流媒体服务器体系结构,并介绍了该系统的工作原理和数据流程。
③在可扩展的流媒体服务器体系结构基础上,对分布式调度机制进行研究,并对负载均衡模块、接入许可控制机制、服务器调度算法进行了改进,提出了基于用户级别的接入许可控制机制和基于容量规划的接入控制优化策略,并实现了动态调度反馈机制。
④针对节点流媒体服务器在系统中的具体功能作用,对服务器的数据存储机制和流媒体服务策略等方面进行了进一步的研究,提出了基于节目流行度的数据存储策略和基于门限的流媒体服务器策略,来提高系统对媒体服务用户的支持能力。
⑤对系统的并发用户数和系统流量输出两个方面进行了测试,本文提出的可扩展流媒体服务器系统体系结构的确扩展了单台流媒体服务器的服务能力,而且具有较好的负载均衡性能。
本文提出的开放体系下可扩展流媒体服务器系统可以更好地发挥流媒体服务器系统中软件和硬件的综合服务能力,是一种解决小规模媒体服务应用的较好方案。该结构搭建比较简单,也不需要较高费用来支持,有较好的性价比,同时具有良好的可扩展性和节点透明性。
Along with the rapid evolution of the Internet, streaming media service such as IPTV and video-on-demand becomes more and more popular, so we need a high performance and flexible streaming server and system to support these services. The Real、Microsoft and Apple companies programs require high-cost support. According to the understanding and analysis of streaming media technology and product, we present a scalable streaming server system to expand media service based on open-frame. By using some of the existing open source projects, we can improve the streaming media server performance and scalability.
The main content is as follows:
①This paper at first introduces the streaming media technology development and the cluster technology. Then it proposes the requirement of streaming media server itself and streaming media service system.
②After detailed analysis of LVS and Darwin server systems, we design a scalable streaming media server architecture based on the two open source projects. Then it introduces its working principle and data flowing.
③Based on the expansion of the streaming media server architecture, the paper The paper proposes the permission control strategy based on user priority and the permission algorithm based on the capacity control and the dynamic feedback scheduling mechanism.
④According to the function of streaming media sever, the paper improves the data storage mechanism and streaming media services strategy. It proposes the storage mechanism based on program popularity and streaming media services strategy based on threshold.
⑤After testing the system concurrent number and the server export, the result indicated the system scalability and load balancing.
The scalable streaming server system improves the services support to users and makes the hardware and software of streaming media server system cooperates much better. It should be a good solution to small-scale media services program.
引文
[1] http://wwv.real.com/
[2] http://www.microsoft.com/
[3] http://www.apple.com/quicktime/
[4] ISO/IEC 14496-15."Information technology-Coding of audio-visual objects-Part 15:AVCFile Format"
[5] Milind Buddhikot. Project MARS: Scalable, high performance, Wen based multimedia-on-demand (MOD) services and servers. Ph.D thesis, Washington University Department of Computer Science, Server Institute of Technology,August 1998
[6] D.C. Thomas. Prospects for interactive Video-on-Demand. IEEE Multimedia.Fall,1994:14-24
[7] Victor O.K.Li, WanJiun Liao. Distributed multimedia systems.Proceedings of the IEEE,1997,85(7):1063-1108
[8] W.D. Sincoskie. System architecture for a large scale video on demand service.Computer Networks and ISDN Systems, 1991:155-162
[9] Mark Podgorny, Geoffrey.Video-on-Demand technologies and demonstrations.Northeast Parallel Architecture Center,March 1997 [10 Yurdaer N. Doganata,Asser N. Tantawi.Making a cost-effective video server.IEEE Multimedia, Winter 1994:22-30
[11] W.Pennebaker, J.Mitchell,.JPEG still image compression standard,.New York Van Nostrand Reinhold, 1993 (12] J.W.Woods, Ed, Subband Image coding. Boston, MA:Kiuwer,1991
[13] A.Gersho, R.M.Gray.Vector quantization and signal compression. Boston,M A:Kiuwer,1992
[14] Joan L.Mitchell. MPEG video compression standard,.New York: Chapman&Hall,1997
[15] ISO/IEC JTC1/SC29/WG11.Overview of the MPEG-4 Standard(N345).Geneva,May/June2000
[16] ISO/IEC JTC1/SC29/WG11,Text of WD 3.0 of MPEG-7 Visual(N3399) Geneva, May/June 2000
[17] ISO/IEC JTC1lSC29/WG1I,MPEG-7 Visual part of XM model Version 6.0,
[18] 曾珂,戴琼海. 流媒体——技术与市场的挑战. ICTC 2002 第十届国际有线电视技术研讨会,2002 年 10 月:10-12
[19] Walid G Aref, Ibrahim Kamel, Niranjan T N. Disk Scheduling for displaying and recording video in non-linear news editing systems, SPIE Multimedia Computing and Networking, 1997, 3020: 228-229
[20] Yueh-Min Huang, Shiao-Li Tsiao, An efficient data placement and retrieval scheme ofzoned-disks to support interactive playout for video servers. IEEE Trans on Consumer Eletronics, 1997,43(1): 69~79
[21] 钟玉琢,向哲,沈洪. 流媒体和视频服务器,北京清华大学出版社,2003.6,214~248
[22] http://www.iozone.org/
[23] ZHU Qing-bo,QIAO Hao,CHEN Dao-xu.Global Memory Management in Distributed Multimedia Storage System, ACTA ELECTRONICA SINICA:152~158
[24] Ozde. A buffer replacement algorithm for multimedia storage system, In Proc of the Third IEEE International Conference on Multimedia Computing System. 1996.172~180
[25] 李德,钱颂迪. 运筹学,北京清华大学出版社 1982
[26] Kai Shen; Tao Yang; Lingkun Chu Cluster load balancing for fine-grain network services Parallel and Distributed Processing Symposium. Proceedings International, IPDPS 2002,Abstracts and CD-ROM , 2002 Page(s): 493 –500
[27] Gilrija Narlikar, Lakshman Y. N., Tin Kam Ho, TaBLA: A Client Based Scheduling Algorithm for Web Proxy Clusters Performance, Computing, and Communications, 2001. IEEE International Conference on. , 2001 Page(s): 217 –227
[28] http:// www.uwsg.iu.edu/hypermail/linux/net/0010.2/0021.html
[29] http://www.tmforum.org/browse.asp?catID=1649
[30] 章文嵩. Linux 服务器集群系统,http://www-900.cn.ibm.com/developerWorks/cn/linux/cluster/lvs/part4/index.shtml#1 , 2002
[31] Sartaj Sahni, Venkat Thanvantri. Performance Metrics: Keeping the Focus on Runtime. IEEE Parallel & Distributed Technology, 1996, 4(1), 43~56
[32] 周佳祥. 异构工作站机群动态平衡系统性能研究,北京:清华大学计算机科学与技术系,1998,2~11
[33] 周佳祥. 机群系统任务调度和负载平衡的研究:北京:清华大学计算机科学与技术系,2000,61~80
[34] L. Borzemski, D. Gajewski, et al. A Load Balancing System for Unix-Based Local Area Networks. Microprocessing and Micro programming, 1993, 39(2), 205~208
[35] Colin Parris. An Authentication Scheme for Use in Dynamic Load Balancing.Technical Report, Computer Science Division (EECS), University of California at Berkeley, 1987
[36] MSDN Library – October 2001/Platform SDK Documentation/Base Services/Performance Monitoring
[37] 康牧,赵治国. 负载调度策略的设计与实现. 控制工程,2003,10(3),219~221
[38] 陈守煜. 系统模糊决策理论与应用. 大连理工大学出版社,1994. 8~21
[39] Paul Russell. Linux netfilter Hacking HOWTO V1.2,2002
[40] Alessandro Rubini, Jonathan Corbet. LINUX 设备驱动程序. 北京:中国电子出版 社,2002,25~62
[41] Scott Maxwell. Linux 内核源代码分析. 机械工业出版社,2000,596~621
[42] Kurt Wall. GNU/Linux 编程指南. 北京:清华大学出版社,2002,229~244
[43] Joseph Mack. LVS-mini-HOWTO. 2002
[2] http://www.microsoft.com/
[3] http://www.apple.com/quicktime/
[4] ISO/IEC 14496-15."Information technology-Coding of audio-visual objects-Part 15:AVCFile Format"
[5] Milind Buddhikot. Project MARS: Scalable, high performance, Wen based multimedia-on-demand (MOD) services and servers. Ph.D thesis, Washington University Department of Computer Science, Server Institute of Technology,August 1998
[6] D.C. Thomas. Prospects for interactive Video-on-Demand. IEEE Multimedia.Fall,1994:14-24
[7] Victor O.K.Li, WanJiun Liao. Distributed multimedia systems.Proceedings of the IEEE,1997,85(7):1063-1108
[8] W.D. Sincoskie. System architecture for a large scale video on demand service.Computer Networks and ISDN Systems, 1991:155-162
[9] Mark Podgorny, Geoffrey.Video-on-Demand technologies and demonstrations.Northeast Parallel Architecture Center,March 1997 [10 Yurdaer N. Doganata,Asser N. Tantawi.Making a cost-effective video server.IEEE Multimedia, Winter 1994:22-30
[11] W.Pennebaker, J.Mitchell,.JPEG still image compression standard,.New York Van Nostrand Reinhold, 1993 (12] J.W.Woods, Ed, Subband Image coding. Boston, MA:Kiuwer,1991
[13] A.Gersho, R.M.Gray.Vector quantization and signal compression. Boston,M A:Kiuwer,1992
[14] Joan L.Mitchell. MPEG video compression standard,.New York: Chapman&Hall,1997
[15] ISO/IEC JTC1/SC29/WG11.Overview of the MPEG-4 Standard(N345).Geneva,May/June2000
[16] ISO/IEC JTC1/SC29/WG11,Text of WD 3.0 of MPEG-7 Visual(N3399) Geneva, May/June 2000
[17] ISO/IEC JTC1lSC29/WG1I,MPEG-7 Visual part of XM model Version 6.0,
[18] 曾珂,戴琼海. 流媒体——技术与市场的挑战. ICTC 2002 第十届国际有线电视技术研讨会,2002 年 10 月:10-12
[19] Walid G Aref, Ibrahim Kamel, Niranjan T N. Disk Scheduling for displaying and recording video in non-linear news editing systems, SPIE Multimedia Computing and Networking, 1997, 3020: 228-229
[20] Yueh-Min Huang, Shiao-Li Tsiao, An efficient data placement and retrieval scheme ofzoned-disks to support interactive playout for video servers. IEEE Trans on Consumer Eletronics, 1997,43(1): 69~79
[21] 钟玉琢,向哲,沈洪. 流媒体和视频服务器,北京清华大学出版社,2003.6,214~248
[22] http://www.iozone.org/
[23] ZHU Qing-bo,QIAO Hao,CHEN Dao-xu.Global Memory Management in Distributed Multimedia Storage System, ACTA ELECTRONICA SINICA:152~158
[24] Ozde. A buffer replacement algorithm for multimedia storage system, In Proc of the Third IEEE International Conference on Multimedia Computing System. 1996.172~180
[25] 李德,钱颂迪. 运筹学,北京清华大学出版社 1982
[26] Kai Shen; Tao Yang; Lingkun Chu Cluster load balancing for fine-grain network services Parallel and Distributed Processing Symposium. Proceedings International, IPDPS 2002,Abstracts and CD-ROM , 2002 Page(s): 493 –500
[27] Gilrija Narlikar, Lakshman Y. N., Tin Kam Ho, TaBLA: A Client Based Scheduling Algorithm for Web Proxy Clusters Performance, Computing, and Communications, 2001. IEEE International Conference on. , 2001 Page(s): 217 –227
[28] http:// www.uwsg.iu.edu/hypermail/linux/net/0010.2/0021.html
[29] http://www.tmforum.org/browse.asp?catID=1649
[30] 章文嵩. Linux 服务器集群系统,http://www-900.cn.ibm.com/developerWorks/cn/linux/cluster/lvs/part4/index.shtml#1 , 2002
[31] Sartaj Sahni, Venkat Thanvantri. Performance Metrics: Keeping the Focus on Runtime. IEEE Parallel & Distributed Technology, 1996, 4(1), 43~56
[32] 周佳祥. 异构工作站机群动态平衡系统性能研究,北京:清华大学计算机科学与技术系,1998,2~11
[33] 周佳祥. 机群系统任务调度和负载平衡的研究:北京:清华大学计算机科学与技术系,2000,61~80
[34] L. Borzemski, D. Gajewski, et al. A Load Balancing System for Unix-Based Local Area Networks. Microprocessing and Micro programming, 1993, 39(2), 205~208
[35] Colin Parris. An Authentication Scheme for Use in Dynamic Load Balancing.Technical Report, Computer Science Division (EECS), University of California at Berkeley, 1987
[36] MSDN Library – October 2001/Platform SDK Documentation/Base Services/Performance Monitoring
[37] 康牧,赵治国. 负载调度策略的设计与实现. 控制工程,2003,10(3),219~221
[38] 陈守煜. 系统模糊决策理论与应用. 大连理工大学出版社,1994. 8~21
[39] Paul Russell. Linux netfilter Hacking HOWTO V1.2,2002
[40] Alessandro Rubini, Jonathan Corbet. LINUX 设备驱动程序. 北京:中国电子出版 社,2002,25~62
[41] Scott Maxwell. Linux 内核源代码分析. 机械工业出版社,2000,596~621
[42] Kurt Wall. GNU/Linux 编程指南. 北京:清华大学出版社,2002,229~244
[43] Joseph Mack. LVS-mini-HOWTO. 2002