基于Linux的流量控制系统的研究与实现
摘要
近年来,随着计算机网络技术的飞速发展,各种综合服务应用越来越广泛,导致带宽需求与日俱增,时常造成网络拥塞。各种分布式多媒体应用不但对网络有很高的带宽要求,而且要求信息传输具有低延迟和低抖动等。一些非关键应用业务,特别是随着eMule、BT、迅雷等各种P2P应用的流行,P2P网络流量已成为互联网的重要组成部分,消耗了大部分的网络带宽资源,严重影响了HTTP、E-mail等其他关键应用的服务质量。如何有效地控制网络流量已成为目前面临的一个重要难题。在带宽资源有限的情况下,若能采用流量控制技术将带宽资源进行有效分配和管理,确保使用者基本的带宽,抑制非关键业务的流量,能大大提升网络的服务质量。
本文深入研究了Linux防火墙Netfilter的框架结构及其工作原理,分析了Netfilter防火墙中的链接跟踪技术及应用层识别分类器L7-filter,并且研究了Linux流量控制的相关技术,对流量控制常用的FIFO、TBF、SFQ、CBQ、HTB等排队规则进行了重点分析。本文基于Linux操作系统的Netfilter防火墙和连接跟踪机制,结合应用层协议识别工具L7-filter和流量控制器TC设计并实现了一个简单而高效的流量控制原型系统。该系统分为流量识别和流量控制两个功能模块,流量识别模块使用iptables和L7-filter识别数据包的协议,然后用iptables对识别出的不同用户组和协议的数据包打上标记进行分类,流量控制模块根据应用需求,对不同用户组和应用协议分配不同的最小保证带宽和最大可借用带宽以及借用优先级,抑制P2P等非关键业务流量,以保证SSH、Telnet、HTTP等关键业务的服务质量。实验证明此系统能有效地控制网络流量,极大地提高了网络的服务质量,而且设备成本低、容易架设,对中小型网络来说是一个很好的QoS解决方案。
In recent years, with the rapid development of computer network technology, a variety of integrated services are used more and more widely, which lead the increasing demand of bandwith. A variety of distributed multimedia applications on the network not only has high bandwidth requirements, and requires information transmission with low latency and low jitter. Some non-critical business applications, especially with the eMule, BT, Thunder and other popular P2P applications, P2P traffic has taken great portions in the network traffic. It is a serious influence to the rest of network service that peer-to-peer flows occupy the network bandwidth seriously, therefore how to effectively identify and control network traffic has been a very important problem.The bandwidth management techniques can make effective management and allocation for the resources of bandwidth, and promote Quality of Service (QoS). Further more, it can control unusual traffic of bandwidth and allocate proper bandwidth.
This thesis studies the Linux firewall and the Netfilter framework works, analysis of the link tracking technology and application layer identification classifier L7-filter in Netfilter firewall and the Linux traffic control technology, FIFO, TBF, SFQ, CBQ, HTB and other queuing rules were focused on analysis.
A simple and efficient traffic control system was designed and realized based on Linux Netfilter firewall and connecting tracking strategy. It combined the L7-filter which identifies packets based on application layer data and TC which is a traffic control tool. The system can identify and classify packets based on application layer data and allocate limited network bandwidth properly to realize traffic control. The experiment result shows that the system can identify and control network traffic effectively. This design way can reduce the abnormal network traffic effectively and then guarantees each user’s basic bandwidth requirement. So the design model is efficient, low cost and easy to build on network which bandwidth needed to be allocated fairly.
本文深入研究了Linux防火墙Netfilter的框架结构及其工作原理,分析了Netfilter防火墙中的链接跟踪技术及应用层识别分类器L7-filter,并且研究了Linux流量控制的相关技术,对流量控制常用的FIFO、TBF、SFQ、CBQ、HTB等排队规则进行了重点分析。本文基于Linux操作系统的Netfilter防火墙和连接跟踪机制,结合应用层协议识别工具L7-filter和流量控制器TC设计并实现了一个简单而高效的流量控制原型系统。该系统分为流量识别和流量控制两个功能模块,流量识别模块使用iptables和L7-filter识别数据包的协议,然后用iptables对识别出的不同用户组和协议的数据包打上标记进行分类,流量控制模块根据应用需求,对不同用户组和应用协议分配不同的最小保证带宽和最大可借用带宽以及借用优先级,抑制P2P等非关键业务流量,以保证SSH、Telnet、HTTP等关键业务的服务质量。实验证明此系统能有效地控制网络流量,极大地提高了网络的服务质量,而且设备成本低、容易架设,对中小型网络来说是一个很好的QoS解决方案。
In recent years, with the rapid development of computer network technology, a variety of integrated services are used more and more widely, which lead the increasing demand of bandwith. A variety of distributed multimedia applications on the network not only has high bandwidth requirements, and requires information transmission with low latency and low jitter. Some non-critical business applications, especially with the eMule, BT, Thunder and other popular P2P applications, P2P traffic has taken great portions in the network traffic. It is a serious influence to the rest of network service that peer-to-peer flows occupy the network bandwidth seriously, therefore how to effectively identify and control network traffic has been a very important problem.The bandwidth management techniques can make effective management and allocation for the resources of bandwidth, and promote Quality of Service (QoS). Further more, it can control unusual traffic of bandwidth and allocate proper bandwidth.
This thesis studies the Linux firewall and the Netfilter framework works, analysis of the link tracking technology and application layer identification classifier L7-filter in Netfilter firewall and the Linux traffic control technology, FIFO, TBF, SFQ, CBQ, HTB and other queuing rules were focused on analysis.
A simple and efficient traffic control system was designed and realized based on Linux Netfilter firewall and connecting tracking strategy. It combined the L7-filter which identifies packets based on application layer data and TC which is a traffic control tool. The system can identify and classify packets based on application layer data and allocate limited network bandwidth properly to realize traffic control. The experiment result shows that the system can identify and control network traffic effectively. This design way can reduce the abnormal network traffic effectively and then guarantees each user’s basic bandwidth requirement. So the design model is efficient, low cost and easy to build on network which bandwidth needed to be allocated fairly.
引文
[1]林闯,单志广,任丰原.计算机网络的服务质量(QoS) [M].北京:清华大学出版社, 2004.4.
[2] Shenker,S.,Partridge,C.,and R.Guerin,Specification of the Guaranteed Quality of Service, RFC 2212,1997.
[3] ISO/IEC. Quality of Service Framework . ISO/IEC JTC 1/SC 21 N9680, CD 13236.2, July1995.
[4] IETF Working Group on Integrated Services[EB/OL]. http://www.ietf.org/html.charters/intserv-charter.html.
[5]冯建业.网络协议分析与流量控制的研究与实现[D].南开大学, 2009.
[6]王恺. MoIP网关QoS模块设计与实现[D].华中科技大学, 2007.
[7]张继军,高鹏.基于分组网络的服务质量保证[M].北京:北京邮电大学出版社, 2004.5.
[8]刘韵洁,张云勇,张智江.下一代网络服务质量技术[M].北京:电子工业出版社, 2005.8.
[9]张国清. QoS在IOS中的实现与应用[M].北京:电子工业出版社, 2010.
[10]徐昌彪,鲜永菊.计算机网络中的拥塞控制与流量控制[M].北京:人民邮电出版社, 2007.10.
[11]郭忠义.分布式并行防火墙系统研究与设计[D].电子科技大学, 2004.
[12]李明. Linux的带宽管理技术研究与应用[D].电子科技大学, 2009.
[13]博嘉科技. Linux防火墙技术探秘[M].北京:国防工业出版社, 2002.10.
[14] You-Sin Yang, Huai-Jen Liu. User-Level Packet Filter for the IP Address Abuse/Counterfeit Problem in Wireless/Wired Networks [C]. Mobile Ad-hoc and Sensor Networks (MSN), 2010 Sixth International Conference, 2010: 239–242.
[15] The netfilter.org project [EB/OL]. http://netfilter.org/, 2010.
[16] Chu-Sing Yang, Ming-Yi Liao, Mon-Yen Luo, Shin-Min Wang. A Network Management System Based on DPI [C]. Network-Based Information Systems (NBiS), 2010 13th International Conference, 2010: 385-388.
[17] Jian Wang, Fuxiang Gao, Guang Zhao. Design and implementation of IPv6-supported network access control gateway [C]. Communication Software and Networks (ICCSN), 2011 IEEE 3rd International Conference, 2011: 522-525
[18] Rusty Russell,Harald Welte.Linux netfilter Hacking HOWTO [EB/OL]. http://www.netfilter.org/documentation/HOWTO/it/netfilter-hacking-HOWTO.html, 2004.
[19] OskarAnderson. Iptables Tutorrial 1.2.2 [EB/OL]. http://iptables-tutorial.frozentux.net/iptables-tutorial.html
[20] Steve Suehring, Robert L. Ziegler. Linux防火墙[M].北京:机械工业出版社, 2006.
[21] P.N.Ayuso. Netfilter's Connection Tracking System [J]. LOGIN;, The USENIX magazine, 2006, 32(3):34--39.
[22] Bing-Heng Peng, Huai-Jen Liu, Huan-Yun Wei. Performance Improvement over Linux Layer-7 Content Filtering [C]. 2009 Fifth International Conference on Mobile Ad-hoc and Sensor Networks. Fujian: IEEE Computer Society, 2009: 522?527.
[23] Application Layer Packet Classifier for Linux [EB/OL]. http://l7-filter.sourceforge.net/.
[24] Bing-Heng Peng, Huai-Jen Liu, Huan-Yun Wei. Performance Improvement over Linux Layer-7 Content Filtering [C]. 2009 Fifth International Conference on Mobile Ad-hoc and Sensor Networks. Fujian: IEEE Computer Society, 2009: 522?527.
[25] M.Othman, M.N.Kermanian. Detecting and preventing peer-to-peer connections by Linux iptables [Z]. International Symposium on Information Technology. Kuala Lumpur, Malaysia: 2008, Vol.4: 1?6.
[26] MartinA.Brown.Traffic Control HOWTO[EB/OL]. http://www.linuxsecurity.com/docs/LDP/Traffic-Control-HOWTO
[27] Almesberger W. Linux Network Traffic Control-Implementation Overview. White Paper, 2001.
[28]赵新.透明网络带宽管理系统的研究与实现[D].南开大学, 2005.
[29]刘勇.网络安全防护体系中QOS流量控制的研究与设计[D].四川大学, 2005.
[30]吴圣杰.基于Linux限制BitTorrent流量的研究与设计[D].北京化工大学, 2007.
[31] B.Hubert. Linux advanced routing & traffic control HOWTO[EB/OL]. http://lartc.org/, 2004.
[32] Sameer Seth, M. Ajaykumar Venkatesulu. TCP/IP架构、设计及应用:Linux版(黄清元,于杰译) [M].北京:清华大学出版社, 2010.
[33]郭献力.基于Linux系统TC框架的带宽控制研究与在校园网中的应用[D].第一军医大学, 2004.
[34]郑盼.多维队列调度排队控制模块的改进研究[D].中山大学, 2010.
[35] Tim Szigeti, Christina Hattingh.端到端QoS网络设计[M].北京:人民邮电出版社, 2006.
[36] Jason Boxman. A Practical Guide to Linux Traffic Control [EB/OL]. http://blog.edseek.com/~jasonb/articles/traffic_shaping/, 2005.
[37] Martin Devera. Hierarchical token buckt [EB/OL]. http://luxik.cdi.cz/~devik/qos/htb, 2003
[38] Martin Devera. Hierachical token bucket theory[EB/OL]. http://luxik.cdi.cz/~devik/qos/htb/manual/theory.htm, 2002
[39] Konvo.HTB description[C],SLT2002,2002.
[40] Stef Coene [EB/OL].http://www.docum.org/docum.org/docs/htb/,
[41] J.L.Valenzuela,A.Monleon,I.San Esteban,et al.A hierarchical token bucket algorithm to enhance QoS in IEEE 802.11:proposal,implementation and evaluation.Vehicular Technology Conference IEEE,2004,60(4):2659-2662
[42]刘芳.网络流量监测与控制[M].北京:北京邮电大学出版社, 2009.
[43]张五生,郑灵翔.基于Linux的流量控制系统研究[J].厦门大学学报(自然科学版), 2010, 49(1):38--42.
[2] Shenker,S.,Partridge,C.,and R.Guerin,Specification of the Guaranteed Quality of Service, RFC 2212,1997.
[3] ISO/IEC. Quality of Service Framework . ISO/IEC JTC 1/SC 21 N9680, CD 13236.2, July1995.
[4] IETF Working Group on Integrated Services[EB/OL]. http://www.ietf.org/html.charters/intserv-charter.html.
[5]冯建业.网络协议分析与流量控制的研究与实现[D].南开大学, 2009.
[6]王恺. MoIP网关QoS模块设计与实现[D].华中科技大学, 2007.
[7]张继军,高鹏.基于分组网络的服务质量保证[M].北京:北京邮电大学出版社, 2004.5.
[8]刘韵洁,张云勇,张智江.下一代网络服务质量技术[M].北京:电子工业出版社, 2005.8.
[9]张国清. QoS在IOS中的实现与应用[M].北京:电子工业出版社, 2010.
[10]徐昌彪,鲜永菊.计算机网络中的拥塞控制与流量控制[M].北京:人民邮电出版社, 2007.10.
[11]郭忠义.分布式并行防火墙系统研究与设计[D].电子科技大学, 2004.
[12]李明. Linux的带宽管理技术研究与应用[D].电子科技大学, 2009.
[13]博嘉科技. Linux防火墙技术探秘[M].北京:国防工业出版社, 2002.10.
[14] You-Sin Yang, Huai-Jen Liu. User-Level Packet Filter for the IP Address Abuse/Counterfeit Problem in Wireless/Wired Networks [C]. Mobile Ad-hoc and Sensor Networks (MSN), 2010 Sixth International Conference, 2010: 239–242.
[15] The netfilter.org project [EB/OL]. http://netfilter.org/, 2010.
[16] Chu-Sing Yang, Ming-Yi Liao, Mon-Yen Luo, Shin-Min Wang. A Network Management System Based on DPI [C]. Network-Based Information Systems (NBiS), 2010 13th International Conference, 2010: 385-388.
[17] Jian Wang, Fuxiang Gao, Guang Zhao. Design and implementation of IPv6-supported network access control gateway [C]. Communication Software and Networks (ICCSN), 2011 IEEE 3rd International Conference, 2011: 522-525
[18] Rusty Russell,Harald Welte.Linux netfilter Hacking HOWTO [EB/OL]. http://www.netfilter.org/documentation/HOWTO/it/netfilter-hacking-HOWTO.html, 2004.
[19] OskarAnderson. Iptables Tutorrial 1.2.2 [EB/OL]. http://iptables-tutorial.frozentux.net/iptables-tutorial.html
[20] Steve Suehring, Robert L. Ziegler. Linux防火墙[M].北京:机械工业出版社, 2006.
[21] P.N.Ayuso. Netfilter's Connection Tracking System [J]. LOGIN;, The USENIX magazine, 2006, 32(3):34--39.
[22] Bing-Heng Peng, Huai-Jen Liu, Huan-Yun Wei. Performance Improvement over Linux Layer-7 Content Filtering [C]. 2009 Fifth International Conference on Mobile Ad-hoc and Sensor Networks. Fujian: IEEE Computer Society, 2009: 522?527.
[23] Application Layer Packet Classifier for Linux [EB/OL]. http://l7-filter.sourceforge.net/.
[24] Bing-Heng Peng, Huai-Jen Liu, Huan-Yun Wei. Performance Improvement over Linux Layer-7 Content Filtering [C]. 2009 Fifth International Conference on Mobile Ad-hoc and Sensor Networks. Fujian: IEEE Computer Society, 2009: 522?527.
[25] M.Othman, M.N.Kermanian. Detecting and preventing peer-to-peer connections by Linux iptables [Z]. International Symposium on Information Technology. Kuala Lumpur, Malaysia: 2008, Vol.4: 1?6.
[26] MartinA.Brown.Traffic Control HOWTO[EB/OL]. http://www.linuxsecurity.com/docs/LDP/Traffic-Control-HOWTO
[27] Almesberger W. Linux Network Traffic Control-Implementation Overview. White Paper, 2001.
[28]赵新.透明网络带宽管理系统的研究与实现[D].南开大学, 2005.
[29]刘勇.网络安全防护体系中QOS流量控制的研究与设计[D].四川大学, 2005.
[30]吴圣杰.基于Linux限制BitTorrent流量的研究与设计[D].北京化工大学, 2007.
[31] B.Hubert. Linux advanced routing & traffic control HOWTO[EB/OL]. http://lartc.org/, 2004.
[32] Sameer Seth, M. Ajaykumar Venkatesulu. TCP/IP架构、设计及应用:Linux版(黄清元,于杰译) [M].北京:清华大学出版社, 2010.
[33]郭献力.基于Linux系统TC框架的带宽控制研究与在校园网中的应用[D].第一军医大学, 2004.
[34]郑盼.多维队列调度排队控制模块的改进研究[D].中山大学, 2010.
[35] Tim Szigeti, Christina Hattingh.端到端QoS网络设计[M].北京:人民邮电出版社, 2006.
[36] Jason Boxman. A Practical Guide to Linux Traffic Control [EB/OL]. http://blog.edseek.com/~jasonb/articles/traffic_shaping/, 2005.
[37] Martin Devera. Hierarchical token buckt [EB/OL]. http://luxik.cdi.cz/~devik/qos/htb, 2003
[38] Martin Devera. Hierachical token bucket theory[EB/OL]. http://luxik.cdi.cz/~devik/qos/htb/manual/theory.htm, 2002
[39] Konvo.HTB description[C],SLT2002,2002.
[40] Stef Coene [EB/OL].http://www.docum.org/docum.org/docs/htb/,
[41] J.L.Valenzuela,A.Monleon,I.San Esteban,et al.A hierarchical token bucket algorithm to enhance QoS in IEEE 802.11:proposal,implementation and evaluation.Vehicular Technology Conference IEEE,2004,60(4):2659-2662
[42]刘芳.网络流量监测与控制[M].北京:北京邮电大学出版社, 2009.
[43]张五生,郑灵翔.基于Linux的流量控制系统研究[J].厦门大学学报(自然科学版), 2010, 49(1):38--42.