面向海量告警数据的并行处理系统设计与实现
|
摘要
为提高海量告警数据的处理效率,设计并实现一种网络管理中面向海量告警数据的并行处理系统,由采集任务更新模块、采集任务调度模块、采集任务处理模块和系统管理模块构成。系统核心为采集任务调度模块,通过采用带宽资源目标和/或计算资源目标计算的双目标任务调度模式,利用采集机的带宽资源和计算资源,提高海量告警数据的采集效率。该系统可广泛适用于网络数据采集及其它领域。
A mass alarm data oriented parallel processing system for network management was designed and implemented.The system was composed of four modules including acquisition task update module,acquisition task scheduling module,acquisition task processing module,and system management module.The core of the system was the task scheduling module,which used both bandwidth resources and computing resources as targets for scheduling.The number of acquisition tasks that could be processed in parallel in a single acquisition trigger cycle was computed.The acquisition tasks were mapped to the physical processing unit of the data acquisition equipment waiting for processing.The bandwidth resources and the computing resources of the data acquisition equipment were utilized,and the efficiency of data acquisition for mass alarm data was improved.The system can be widely used for network data acquisition and other related areas.
A mass alarm data oriented parallel processing system for network management was designed and implemented.The system was composed of four modules including acquisition task update module,acquisition task scheduling module,acquisition task processing module,and system management module.The core of the system was the task scheduling module,which used both bandwidth resources and computing resources as targets for scheduling.The number of acquisition tasks that could be processed in parallel in a single acquisition trigger cycle was computed.The acquisition tasks were mapped to the physical processing unit of the data acquisition equipment waiting for processing.The bandwidth resources and the computing resources of the data acquisition equipment were utilized,and the efficiency of data acquisition for mass alarm data was improved.The system can be widely used for network data acquisition and other related areas.
引文
[1]YANG Jinbao,ZHANG Changhong,CHEN Ping.Network fault diagnosis based on improved BP neural network[J].Computer&Digital Engineering,2012,40(2):65-67(in Chinese).[杨金宝,张昌宏,陈平.基于改进BP神经网络的网络故障诊断研究[J].计算机与数字工程,2012,40(2):65-67.]
[2]Slabicki M,Grochla K.Performance evaluation of CoAP,SNMP and NETCONF protocols in fog computing architecture[C]//Network Operations and Management Symposium,2016.
[3]Shahram Jamali,Mohammad Sadeq Garshasbi.Fault localization algorithm in computer networks by employing agenetic algorithm[J].Journal of Experimental&Theoretical Artificial Intelligence,2017,29(1):157-174.
[4]SHAO Lei,NI Ming.Design and implementation of high speed data acquisition system based on FPGA[J].Computer Engineering,2011,37(19):221-223(in Chinese).[邵磊,倪明.基于FPGA的高速数据采集系统设计与实现[J].计算机工程,2011,37(19):221-223.]
[5]YS Peng,YC Chen,SNMP-based monitoring of heterogeneous virtual infrastructure in clouds[C]//Network Operations and Management Symposium,2011:1-6.
[6]HU Yi,TANG Liping.Implementation of Telnet server for Non-OS embedded system[J].Computer Systerm&Applications,2014,23(10):79-84(in Chinese).[胡奕,唐莉萍.嵌入式系统上无操作系统Telnet服务器的实现[J].计算机系统应用,2014,23(10):79-84.]
[7]YIN Weihong,GENG Xinmin.The implement of network management based on the SNMP protocol[J].Microcomputer Information,2006,22(9-3):208-210(in Chinese).[殷卫红,耿新民.基于SNMP协议的网络管理实现技术[J].微计算机信息,2006,22(9-3):208-210.]
[8]XIAO Jingjing.Design of network manage system based on ITIL[J].Software Guide,2015,14(10):124-125(in Chinese).[肖静静.基于ITIL的网络运维服务管理系统设计[J].软件导刊,2015,14(10):124-125.]
[9]YANG Ming,HUANG Hua,XIA Jiangang,et al.Current status and analysis on network fault-locating technology[J].Technology and Market,2009,16(3):20-22(in Chinese).[杨明,黄华,夏建刚,等.网络故障定位技术的研究现状与分析[J].技术与市场,2009,16(3):20-22.]
[10]WANG Ping,LI Li,ZHAO Hong.Study on events correlation detecting mechanism in network management[J].Journal of China Institue of Communications,2004,3:100-105(in Chinese).[王平,李莉,赵宏.网络管理中事件关联检测机制的研究[J].通信学报,2004,3:100-105.]
[11]JING Ya,LI Qun,CHEN Ting,et al.Large-scale heteroid network management system[J].Computer Engineering and Design,2015,36(12):3390-3401(in Chinese).[井雅,李群,陈亭,等.大规模异构通信网络管理信息系统[J].计算机工程与设计,2015,36(12):3390-3401.]
[12]JING Ya,CHEN Ting.Electric material supplies management information system based on PDA[J].Computer Engineering and Design,2016,37(4):1113-1121(in Chinese).[井雅,陈亭.基于PDA的电力物资到货管理信息系统[J].计算机工程与设计,2016,37(4):1113-1121.]
[2]Slabicki M,Grochla K.Performance evaluation of CoAP,SNMP and NETCONF protocols in fog computing architecture[C]//Network Operations and Management Symposium,2016.
[3]Shahram Jamali,Mohammad Sadeq Garshasbi.Fault localization algorithm in computer networks by employing agenetic algorithm[J].Journal of Experimental&Theoretical Artificial Intelligence,2017,29(1):157-174.
[4]SHAO Lei,NI Ming.Design and implementation of high speed data acquisition system based on FPGA[J].Computer Engineering,2011,37(19):221-223(in Chinese).[邵磊,倪明.基于FPGA的高速数据采集系统设计与实现[J].计算机工程,2011,37(19):221-223.]
[5]YS Peng,YC Chen,SNMP-based monitoring of heterogeneous virtual infrastructure in clouds[C]//Network Operations and Management Symposium,2011:1-6.
[6]HU Yi,TANG Liping.Implementation of Telnet server for Non-OS embedded system[J].Computer Systerm&Applications,2014,23(10):79-84(in Chinese).[胡奕,唐莉萍.嵌入式系统上无操作系统Telnet服务器的实现[J].计算机系统应用,2014,23(10):79-84.]
[7]YIN Weihong,GENG Xinmin.The implement of network management based on the SNMP protocol[J].Microcomputer Information,2006,22(9-3):208-210(in Chinese).[殷卫红,耿新民.基于SNMP协议的网络管理实现技术[J].微计算机信息,2006,22(9-3):208-210.]
[8]XIAO Jingjing.Design of network manage system based on ITIL[J].Software Guide,2015,14(10):124-125(in Chinese).[肖静静.基于ITIL的网络运维服务管理系统设计[J].软件导刊,2015,14(10):124-125.]
[9]YANG Ming,HUANG Hua,XIA Jiangang,et al.Current status and analysis on network fault-locating technology[J].Technology and Market,2009,16(3):20-22(in Chinese).[杨明,黄华,夏建刚,等.网络故障定位技术的研究现状与分析[J].技术与市场,2009,16(3):20-22.]
[10]WANG Ping,LI Li,ZHAO Hong.Study on events correlation detecting mechanism in network management[J].Journal of China Institue of Communications,2004,3:100-105(in Chinese).[王平,李莉,赵宏.网络管理中事件关联检测机制的研究[J].通信学报,2004,3:100-105.]
[11]JING Ya,LI Qun,CHEN Ting,et al.Large-scale heteroid network management system[J].Computer Engineering and Design,2015,36(12):3390-3401(in Chinese).[井雅,李群,陈亭,等.大规模异构通信网络管理信息系统[J].计算机工程与设计,2015,36(12):3390-3401.]
[12]JING Ya,CHEN Ting.Electric material supplies management information system based on PDA[J].Computer Engineering and Design,2016,37(4):1113-1121(in Chinese).[井雅,陈亭.基于PDA的电力物资到货管理信息系统[J].计算机工程与设计,2016,37(4):1113-1121.]