内网监控系统中监控中心服务器的设计与实现
|
摘要
随着计算机技术的飞速发展以及网络规模的日益扩大,网络逐渐深入到各行业领域和人们的生活中。如今,信息社会已经离不开网络,这就使得网络自身运行的稳定性和可靠性成为不容忽视的重要因素。为了提高对网络的管理性,网络监控成为一种必要的手段。通过使用网络监控系统,人们可以及时获取到网络中各类被监测实体对象的性能数据,从而分析它们的运行状况,及时发现相关问题。这给网络管理员提供了一种方便,高效的管理方式。
本文将内网监控系统中的监控中心服务器作为研究对象。内网监控系统作为全文的项目背景,综合了集中式监控和分布式监控的特点。它围绕集中管理,分布监控的设计思想,通过在网络中部署多台监控代理实现全网范围的监控。而监控中心服务器作为控制中心,它是系统的重要组成部分。在具体实现过程中,我们采用模块化设计方法将监控中心服务器划分为六个不同的功能模块,其中,前三个模块体现了中心服务器的核心功能。概括来讲,全文主要工作内容包括:
(一)分析了监控中心服务器实现时所用到的一些关键技术,并对服务器的总体框架和模块结构进行设计及主要功能实现。
(二)依据监控中心服务器自身的通信特点,通过采用TCP协议和统一的报文头,设计出一套完整的系统通信机制,用来实现监控中心服务器与代理端之间的信息交互。
(三)针对监控中心服务器的并发性通信问题,设计一种线程池与队列相结合的处理方法。通过利用多线程去主动处理和接受同时来自多个代理端的采集数据,从而提高系统整体并发性能。
(四)设计一种带缓冲区的存储处理方法,目的是优化数据存储效率。首先,通过工作线程把所有接受到的采集数据集中存入缓冲区内,然后由单独的线程定期去从中取数据来完成整个存储操作。
最后,我们对监控中心服务器进行多方面测试,结果表明,系统在满足功能需求的前提下,能够在整体上表现出较好的运行性能。
With the rapid development of computer technology, as well as the growing size of the network, the network has spread into various industries and people's lives. Today, information society becomes inseparable from the network. And the stability of network and reliability of its operation has become important factors that can not be neglected. In order to improve network management, network monitoring can be a necessary means. With the help of network monitoring system, we can acquire a variety of performance data and analyze the operating conditions, so the problems could be detected timely. This provides network administrators with a convenient and efficient way of management.
We mainly study the center server of the inner-network monitoring system in this thesis. As the project background, the inner-network monitoring system integrates the merits of centralized monitoring and distributed monitoring. And it focuses on the idea of centralized management and distributed monitoring. So network monitoring can be achieved by deploying multiple monitoring agents. Since the monitoring center server takes the role of control center, it is an important part of the whole system. In the process of realization, we divide the structure of server into six modules based on the principle of modular design. And the first three modules reflect the core function of server. Generally speaking, we mainly do the following work in the thesis:
1. We analyze some key technologies in implementing the monitoring center server, and then design the structure of monitoring center server and its function modules.
2. According to the communication features, we work out a set of communications rules by adopting TCP protocol and a regular head of message. It is used to achieve interaction between the server and agent.
3. For the concurrency issues in communication, we design a special method. The feature of it is that we could handle collected data from multiple agents by introducing multi-threads at the same time. So it can increase overall system capacity of parallel processing.
4. A storing approach with a buffer is designed for the purpose of optimizing the efficiency in storing data. Firstly, all collected data are received into a buffer by worker threads. Then, a special thread would regularly get data from the buffer to achieve the storage.
Finally, we test the server in many aspects. And the results show that the system can effectively run under the premise of realizing function.
本文将内网监控系统中的监控中心服务器作为研究对象。内网监控系统作为全文的项目背景,综合了集中式监控和分布式监控的特点。它围绕集中管理,分布监控的设计思想,通过在网络中部署多台监控代理实现全网范围的监控。而监控中心服务器作为控制中心,它是系统的重要组成部分。在具体实现过程中,我们采用模块化设计方法将监控中心服务器划分为六个不同的功能模块,其中,前三个模块体现了中心服务器的核心功能。概括来讲,全文主要工作内容包括:
(一)分析了监控中心服务器实现时所用到的一些关键技术,并对服务器的总体框架和模块结构进行设计及主要功能实现。
(二)依据监控中心服务器自身的通信特点,通过采用TCP协议和统一的报文头,设计出一套完整的系统通信机制,用来实现监控中心服务器与代理端之间的信息交互。
(三)针对监控中心服务器的并发性通信问题,设计一种线程池与队列相结合的处理方法。通过利用多线程去主动处理和接受同时来自多个代理端的采集数据,从而提高系统整体并发性能。
(四)设计一种带缓冲区的存储处理方法,目的是优化数据存储效率。首先,通过工作线程把所有接受到的采集数据集中存入缓冲区内,然后由单独的线程定期去从中取数据来完成整个存储操作。
最后,我们对监控中心服务器进行多方面测试,结果表明,系统在满足功能需求的前提下,能够在整体上表现出较好的运行性能。
With the rapid development of computer technology, as well as the growing size of the network, the network has spread into various industries and people's lives. Today, information society becomes inseparable from the network. And the stability of network and reliability of its operation has become important factors that can not be neglected. In order to improve network management, network monitoring can be a necessary means. With the help of network monitoring system, we can acquire a variety of performance data and analyze the operating conditions, so the problems could be detected timely. This provides network administrators with a convenient and efficient way of management.
We mainly study the center server of the inner-network monitoring system in this thesis. As the project background, the inner-network monitoring system integrates the merits of centralized monitoring and distributed monitoring. And it focuses on the idea of centralized management and distributed monitoring. So network monitoring can be achieved by deploying multiple monitoring agents. Since the monitoring center server takes the role of control center, it is an important part of the whole system. In the process of realization, we divide the structure of server into six modules based on the principle of modular design. And the first three modules reflect the core function of server. Generally speaking, we mainly do the following work in the thesis:
1. We analyze some key technologies in implementing the monitoring center server, and then design the structure of monitoring center server and its function modules.
2. According to the communication features, we work out a set of communications rules by adopting TCP protocol and a regular head of message. It is used to achieve interaction between the server and agent.
3. For the concurrency issues in communication, we design a special method. The feature of it is that we could handle collected data from multiple agents by introducing multi-threads at the same time. So it can increase overall system capacity of parallel processing.
4. A storing approach with a buffer is designed for the purpose of optimizing the efficiency in storing data. Firstly, all collected data are received into a buffer by worker threads. Then, a special thread would regularly get data from the buffer to achieve the storage.
Finally, we test the server in many aspects. And the results show that the system can effectively run under the premise of realizing function.
引文
[1] C.Hopp, S.Stoll, U.Konigorski. Remote control design and implementation using the internet. Proceedings of 5th Biannual World Automation Congress, 2002, 14:481-486
[2] S.H.Yang, C.W.Dai, R.E Knott. Remote maintenance of control system performance over the interact. Control Engineering Practice, 2007, 15(5):533-544
[3] W.E.Chang, Y.C.Wu, C.W.Chiu. Development of a web-based remote 1oad supervision and control system. Electrical Power & Energy Systems, 2006, 28(6):401-407
[4] H.Ewald, G.F.Page. Performing experiments by remote control using the intemet. Global Journal of Engineering Education, 2000, 4(3):287-292
[5] S.H.Yang, L.S.Tan,X.Chen.Requirements specification and architecture design for internet-based control systems. Proceedings of the 26th International Computer Software and Applications Conference, 2002:75-80
[6]李玉珠.基于Web的炼焦生产实时监控系统设计与实现:[硕士学位论文].长沙:中南大学,2009:8-20
[7]王丰贵.因特网环境下分布式监测系统关键技术研究及应用:[博士学位论文].长春:吉林大学,2009:19-45
[8]袁立成.基于XML的海洋环境信息数据格式转换:[硕士学位论文].青岛:中国海洋大学,2009:5-15
[9] S.H.Yang, X.Chen, J.L.Alty. Design issues and implementation of intemet-based process control systems. Control Engineering Practice, 2003, 11(6):709-720
[10] H.T.Yang, R.Eagleson. Design and implementation of an intemet-based embedded control system. Proceedings of 2003 IEEE Conference on Control Applications, 2003, 2:1181-1185
[11]姜毅.一种新型基于HTTP的实时信息传输方法的研究:[硕士学位论文].杭州:浙江理工大学,2008:6-18
[12] S.K.Ong, W.W.Sun. Application of mobile agents in a web-based real-time monitoring system. International Joumal of Advanced Manufacturing Technology, 2003, 22(1-2):33-40
[13] M.Furuya, H.Kato, T.Sekozawa. Secure web-based monitoring and control system. IECON Proceedings, 2000, 4:2443-2448
[14] W.Egwin, Y.Leena, J.Pasi. Web based monitoring and control of industrial processes. University of Oulu, Control Engineering Laboratory, 2003:411-415
[15]姚益圣,马增良,邹益仁.一个安全可靠的远程组态监控系统的设计与实现.计算机工程与应用,2003,4:219-221
[16]解娇.网络监控客户端系统的设计与实现: [硕士学位论文].大连:大连理工大学,2008:11-23
[17] Wu Fang, Zhao Zhijin, Ye Xueyi. A New Dynamic Network Monitoring Based On IA. International Conference on Computer Science and Computational Technology, 2008, Vol.2:637-640
[18] Politopoulos, P.I., Markatos, E.P., Ioannidis, S.. Evaluation of Compression of Remote Network Monitoring Data Streams. Network Operations and Management Symposium Workshops, 2008, 4:109-115
[19] Jiang Chunmao. The Software Design of the Network Monitoring Device Based on Linux System Platform. International Conference on Education Technology and Computer, 2009, 4:262-264
[20] Ligocki, N.P., Hara, C.S., Lyra, C.. A flexible network monitoring tool based on a data stream management system. International Conference on Computers and Communications, 2008, 7:800-805
[21] Wang Xuren, He Famei. An Implement of Broadband Network Monitoring System Based on Libnids and Winpcap. International Conference on New Trends in Information and Service Science, 2009, 7:812-814
[22] Pugazendi, R., Duraiswamy, K.. Mobile Agents - A Solution for Network Monitoring. International Conference on Advances in Recent Technologies in Communication and Computing, 2009, 8:579-584
[23] Hu Xiangdong, Gao Zhan; Li Wei. Research on the Switched LAN Monitor Mechanism and its Implementation Method Based on ARP Spoofing. International Conference on Management and Service Science, 2009, 9:1-4
[24] Xiao Wei, Wei Wu, Yunxiang Liu. A Network Monitor System Model with Performance Feedback Function. International Conference on E-Business and Information System Security, 2009, 5:1-5
[25] Sasaki, M.; Liang Zhao; Nagamochi, H.. Security-aware beacon based network monitoring. International Conference on Communication Systems, 2008, 11:527-531
[26] Valliyammai, C., Selvi, S.T.. Relational Network Monitoring System for Grid Performance Optimization. International Conference on Advanced Computing and Communications, 2008, 12:170-173
[27] Yalagandula, P., Sung-Ju Lee, Puneet Sharma, et al. Correlations in end-to-end network metrics: Impact on large scale network monitoring. Computer Communications Workshops, 2008, 4:1-6
[28] Moraes, P.S., Sampaio, L.N., Monteiro, J.A.S., et al.. MonONTO: A Domain Ontology for Network Monitoring and Recommendation for Advanced Internet Applications Users. Network Operations and Management Symposium Workshops, 2008, 4:116-123
[29] Ubik, S., Zejdl, P., Halak, J.. Real-time anonymization in passive network monitoring. International Conference on Networking and Services, 2007, 6:100-100
[30] Misun Yu, Haeyong Kim, Pyeongsoo Mah. NanoMon: An Adaptable Sensor Network Monitoring Software. International Conference on Consumer Electronics, 2007, 6:1-6
[31] Fenggui Wang, Di Tian, Hongsheng Yang, et al. A distributed monitoring system for working status of scientific instruments. International Conference on Automation and Logistics, 2008, 9:1016-1021
[32] Grzech, A. Optimization of Two-Level Topological Structure of Distributed Intrusion Detection System. International Conference on Systems Engineering, 2008, 8: 337-342
[33] Hamlyn, A., Cheung, H., Lin Wang, et al. Distributed monitoring and centralized forecasting network for DG-connected distribution systems. Power and Energy Society General Meeting - Conversion and Delivery of Electrical Energy in the 21st Century, 2008, 7:1-7
[34] Hamlyn, A., Cheung, H., Cungang Yang. Computer network distributed monitoring and centralized forecasting of utility distribution system operations. International Conference on Electrical and Computer Engineering, 2008, 5:1719-1722
[35] Hamlyn, A., Cheung, H., Lin Wang, et al. Distributed Monitoring and Centralized Forecasting System for DG-Connected Distribution Systems. International Conference on Power Engineering, 2007, 8:58-62
[36] Gi Heung Choi, Gi Sang Choi. Design of Service System Framework for Web-Based Monitoring and Control of Indoor Air Quality (IAQ) in Subway Stations. International Conference on New Trends in Information and Service Science, 2009, 7:378-382
[37]刘勇,潘成胜,莫蓉等.分布式代理技术在监控系统中的应用.火力与指挥控制.2007,32(12):149-152
[38]周大伟,王志锋,许炎义等.一种分布式实时监控系统的设计与实现.计算机与数字工程.2009,37(2):66-68
[39]黄志兴,齐德昱.主机监控系统的分布式控制研究.科学技术与工程.2009,9(7):1739-1740
[40]江玉玲.浅析远程监测系统中的数据传输和数据管理.仪表技术.2009,8:43-45
[41]陈为军.网络监控技术的研究与应用:[硕士学位论文].上海:华东师范大学,2007:9-20
[42]周坤.计算机远程终端监控系统:[硕士学位论文].南京:南京信息工程大学,2008:10-24
[43]刘金花,万明秀.一种新型监控管理系统的研究与实现.微计算机信息.2009,8:74-75
[44]谢伟,冯杰.浅谈多层分布式应用体系结构.科技信息.2008,14:178-179
[45]杨卫东,施伯乐.XML流管理研究综述.计算机研究与发展.2009,46(10):1721-1727
[46]赵冲冲,王鑫,胡长军.XML格式领域数据传输的优化技术研究.计算机科学. 2009,36(8):185-188
[47]蒋永国,刘克,卢莲英等.基于XML的海洋遥感卫星数据规范化研究.海洋通报. 2009,28(4):9-15
[48]胡德敏,李抒娅.基于CORBA、XML的交通综合信息平台的研究与设计.计算机工程与设计.2009,30(4):1000-1002
[49]李静,韦志鹏,田俊松.浅析JDBC在数据库链接中的应用.农业网络信息.2009,9:77-78
[50]李贤华,彭真.基于JDBC-ODBC的数据库访问方法的设计与实现.计算机与现代化. 2008,5:28-31
[51]宋勇,李建辉.基于JDBC的数据库连接池技术的优化及实现.科学技术与工程. 2007,7(23):6077-6078
[2] S.H.Yang, C.W.Dai, R.E Knott. Remote maintenance of control system performance over the interact. Control Engineering Practice, 2007, 15(5):533-544
[3] W.E.Chang, Y.C.Wu, C.W.Chiu. Development of a web-based remote 1oad supervision and control system. Electrical Power & Energy Systems, 2006, 28(6):401-407
[4] H.Ewald, G.F.Page. Performing experiments by remote control using the intemet. Global Journal of Engineering Education, 2000, 4(3):287-292
[5] S.H.Yang, L.S.Tan,X.Chen.Requirements specification and architecture design for internet-based control systems. Proceedings of the 26th International Computer Software and Applications Conference, 2002:75-80
[6]李玉珠.基于Web的炼焦生产实时监控系统设计与实现:[硕士学位论文].长沙:中南大学,2009:8-20
[7]王丰贵.因特网环境下分布式监测系统关键技术研究及应用:[博士学位论文].长春:吉林大学,2009:19-45
[8]袁立成.基于XML的海洋环境信息数据格式转换:[硕士学位论文].青岛:中国海洋大学,2009:5-15
[9] S.H.Yang, X.Chen, J.L.Alty. Design issues and implementation of intemet-based process control systems. Control Engineering Practice, 2003, 11(6):709-720
[10] H.T.Yang, R.Eagleson. Design and implementation of an intemet-based embedded control system. Proceedings of 2003 IEEE Conference on Control Applications, 2003, 2:1181-1185
[11]姜毅.一种新型基于HTTP的实时信息传输方法的研究:[硕士学位论文].杭州:浙江理工大学,2008:6-18
[12] S.K.Ong, W.W.Sun. Application of mobile agents in a web-based real-time monitoring system. International Joumal of Advanced Manufacturing Technology, 2003, 22(1-2):33-40
[13] M.Furuya, H.Kato, T.Sekozawa. Secure web-based monitoring and control system. IECON Proceedings, 2000, 4:2443-2448
[14] W.Egwin, Y.Leena, J.Pasi. Web based monitoring and control of industrial processes. University of Oulu, Control Engineering Laboratory, 2003:411-415
[15]姚益圣,马增良,邹益仁.一个安全可靠的远程组态监控系统的设计与实现.计算机工程与应用,2003,4:219-221
[16]解娇.网络监控客户端系统的设计与实现: [硕士学位论文].大连:大连理工大学,2008:11-23
[17] Wu Fang, Zhao Zhijin, Ye Xueyi. A New Dynamic Network Monitoring Based On IA. International Conference on Computer Science and Computational Technology, 2008, Vol.2:637-640
[18] Politopoulos, P.I., Markatos, E.P., Ioannidis, S.. Evaluation of Compression of Remote Network Monitoring Data Streams. Network Operations and Management Symposium Workshops, 2008, 4:109-115
[19] Jiang Chunmao. The Software Design of the Network Monitoring Device Based on Linux System Platform. International Conference on Education Technology and Computer, 2009, 4:262-264
[20] Ligocki, N.P., Hara, C.S., Lyra, C.. A flexible network monitoring tool based on a data stream management system. International Conference on Computers and Communications, 2008, 7:800-805
[21] Wang Xuren, He Famei. An Implement of Broadband Network Monitoring System Based on Libnids and Winpcap. International Conference on New Trends in Information and Service Science, 2009, 7:812-814
[22] Pugazendi, R., Duraiswamy, K.. Mobile Agents - A Solution for Network Monitoring. International Conference on Advances in Recent Technologies in Communication and Computing, 2009, 8:579-584
[23] Hu Xiangdong, Gao Zhan; Li Wei. Research on the Switched LAN Monitor Mechanism and its Implementation Method Based on ARP Spoofing. International Conference on Management and Service Science, 2009, 9:1-4
[24] Xiao Wei, Wei Wu, Yunxiang Liu. A Network Monitor System Model with Performance Feedback Function. International Conference on E-Business and Information System Security, 2009, 5:1-5
[25] Sasaki, M.; Liang Zhao; Nagamochi, H.. Security-aware beacon based network monitoring. International Conference on Communication Systems, 2008, 11:527-531
[26] Valliyammai, C., Selvi, S.T.. Relational Network Monitoring System for Grid Performance Optimization. International Conference on Advanced Computing and Communications, 2008, 12:170-173
[27] Yalagandula, P., Sung-Ju Lee, Puneet Sharma, et al. Correlations in end-to-end network metrics: Impact on large scale network monitoring. Computer Communications Workshops, 2008, 4:1-6
[28] Moraes, P.S., Sampaio, L.N., Monteiro, J.A.S., et al.. MonONTO: A Domain Ontology for Network Monitoring and Recommendation for Advanced Internet Applications Users. Network Operations and Management Symposium Workshops, 2008, 4:116-123
[29] Ubik, S., Zejdl, P., Halak, J.. Real-time anonymization in passive network monitoring. International Conference on Networking and Services, 2007, 6:100-100
[30] Misun Yu, Haeyong Kim, Pyeongsoo Mah. NanoMon: An Adaptable Sensor Network Monitoring Software. International Conference on Consumer Electronics, 2007, 6:1-6
[31] Fenggui Wang, Di Tian, Hongsheng Yang, et al. A distributed monitoring system for working status of scientific instruments. International Conference on Automation and Logistics, 2008, 9:1016-1021
[32] Grzech, A. Optimization of Two-Level Topological Structure of Distributed Intrusion Detection System. International Conference on Systems Engineering, 2008, 8: 337-342
[33] Hamlyn, A., Cheung, H., Lin Wang, et al. Distributed monitoring and centralized forecasting network for DG-connected distribution systems. Power and Energy Society General Meeting - Conversion and Delivery of Electrical Energy in the 21st Century, 2008, 7:1-7
[34] Hamlyn, A., Cheung, H., Cungang Yang. Computer network distributed monitoring and centralized forecasting of utility distribution system operations. International Conference on Electrical and Computer Engineering, 2008, 5:1719-1722
[35] Hamlyn, A., Cheung, H., Lin Wang, et al. Distributed Monitoring and Centralized Forecasting System for DG-Connected Distribution Systems. International Conference on Power Engineering, 2007, 8:58-62
[36] Gi Heung Choi, Gi Sang Choi. Design of Service System Framework for Web-Based Monitoring and Control of Indoor Air Quality (IAQ) in Subway Stations. International Conference on New Trends in Information and Service Science, 2009, 7:378-382
[37]刘勇,潘成胜,莫蓉等.分布式代理技术在监控系统中的应用.火力与指挥控制.2007,32(12):149-152
[38]周大伟,王志锋,许炎义等.一种分布式实时监控系统的设计与实现.计算机与数字工程.2009,37(2):66-68
[39]黄志兴,齐德昱.主机监控系统的分布式控制研究.科学技术与工程.2009,9(7):1739-1740
[40]江玉玲.浅析远程监测系统中的数据传输和数据管理.仪表技术.2009,8:43-45
[41]陈为军.网络监控技术的研究与应用:[硕士学位论文].上海:华东师范大学,2007:9-20
[42]周坤.计算机远程终端监控系统:[硕士学位论文].南京:南京信息工程大学,2008:10-24
[43]刘金花,万明秀.一种新型监控管理系统的研究与实现.微计算机信息.2009,8:74-75
[44]谢伟,冯杰.浅谈多层分布式应用体系结构.科技信息.2008,14:178-179
[45]杨卫东,施伯乐.XML流管理研究综述.计算机研究与发展.2009,46(10):1721-1727
[46]赵冲冲,王鑫,胡长军.XML格式领域数据传输的优化技术研究.计算机科学. 2009,36(8):185-188
[47]蒋永国,刘克,卢莲英等.基于XML的海洋遥感卫星数据规范化研究.海洋通报. 2009,28(4):9-15
[48]胡德敏,李抒娅.基于CORBA、XML的交通综合信息平台的研究与设计.计算机工程与设计.2009,30(4):1000-1002
[49]李静,韦志鹏,田俊松.浅析JDBC在数据库链接中的应用.农业网络信息.2009,9:77-78
[50]李贤华,彭真.基于JDBC-ODBC的数据库访问方法的设计与实现.计算机与现代化. 2008,5:28-31
[51]宋勇,李建辉.基于JDBC的数据库连接池技术的优化及实现.科学技术与工程. 2007,7(23):6077-6078