一种改进模糊kNN的云计算故障检测方法
|
摘要
在大规模复杂的云计算体系中,通过采集、分析系统数据可以了解系统运行的状态,从而发现并解决云计算故障问题.然而,目前基于监督学习的云计算故障检测方法忽略了噪声数据处理、训练样本的更新、未知类型故障的识别,影响了云计算故障检测的准确性.为此,本文定义了云计算故障模型并提出一种改进模糊k NN的云计算故障检测方法.该方法首先使用基于密度聚类的方法对初始云计算故障数据训练集进行预处理;其次根据模糊熵与互信息相结合的方法对云计算故障特征进行加权;然后根据故障特征权值以及分层检测改进模糊k NN,确定待检测云计算数据的近邻训练样本;最后通过基于最大隶属度的自学习确定待检测云计算数据的故障检测结果.通过实验表明本文方法对云计算故障检测是有效的.
In large-scale and complex cloud computing system,the state of system operation can be gained through collecting and analyzing system data. Then the faults of cloud computing will be found and solved. However,the current methods based on supervised learning ignore the noise data processing,the training samples updating and the identification of unknown type faults,which will affect the accuracy of cloud computing fault detection. In this paper,the cloud computing fault model is defined,and a cloud computing fault detection method improved fuzzy k NN is proposed. Firstly,the initial training set of cloud computing fault data are preprocessed using the method based on density clustering. Secondly,the feature of cloud computing fault is weighted according to the method of combining fuzzy entropy with mutual information. The fuzzy k NN is improved according to the fault feature weight and hierarchical detection,and the nearest neighbor sample of cloud computing data will be detected. Finally,the detection results of cloud computing data are determined by self-learning based on the maximum membership degree. Experimental results showthat this method is effective in fault detection of cloud computing.
In large-scale and complex cloud computing system,the state of system operation can be gained through collecting and analyzing system data. Then the faults of cloud computing will be found and solved. However,the current methods based on supervised learning ignore the noise data processing,the training samples updating and the identification of unknown type faults,which will affect the accuracy of cloud computing fault detection. In this paper,the cloud computing fault model is defined,and a cloud computing fault detection method improved fuzzy k NN is proposed. Firstly,the initial training set of cloud computing fault data are preprocessed using the method based on density clustering. Secondly,the feature of cloud computing fault is weighted according to the method of combining fuzzy entropy with mutual information. The fuzzy k NN is improved according to the fault feature weight and hierarchical detection,and the nearest neighbor sample of cloud computing data will be detected. Finally,the detection results of cloud computing data are determined by self-learning based on the maximum membership degree. Experimental results showthat this method is effective in fault detection of cloud computing.
引文
[1] Zhou Dong-hua,Hu Yan-yan. Fault diagnosis techniques for dynamic systems[J]. Acta Automatica Sinica,2009,35(6):748-758.
[2]Wang Tao,Zhang Wen-bo,Xu Ji-wei,et al. A survey of fault detection for distributed software systems with statistical monitoring in cloud computing[J]. Chinese Journal of Computers,2017,40(2):397-413.
[3] Meng S. Monitoring-as-a-service in the cloud[D]. Georgia Institute of Technology,2012.
[4] Armbrust M,Fox A,Griffith R,et al. A view of cloud computing[J]. Communications of the ACM,2010,53(4):50-58.
[5]Fu Wen-long,Zhou Jian-zhong,Li Chao-shun,et al. Vibrant fault diagnosis for hydro-electric generating unit based on support vector data description improved w ith fuzzy k-nearest neighbor[J]. Proceedings of the CSEE,2014,34(32):5788-5795.
[6]Qing Guan,Zhang Zi-ming,Song Fu. Ensemble of bayesian predictors and decision trees for proactive failure management in cloud computing systems[J]. Journal of Communications,2012,7(1):52-61.
[7]Modi C N,Patel D R,Patel A,et al. Bayesian classifier and snort based netw ork intrusion detection systemin cloud computing[C].Third International Conference on Computing,Communication and Netw orking Technologies(ICCCNT'12),2012:1-7.
[8]Stehle E,Lynch K,Shevertalov M,et al. On the use of computational geometry to detect softw are faults at runtime[C]. Proceedings of the 7th International Conference on Autonomic Computing,ACM,2010:109-118.
[9]Luo Jun-zhou,Jin Jia-hui,Song Ai-bo,et al. Cloud computing:architecture and key technologies[J]. Journal on Communications,2011,32(7):3-21.
[10] Zhang Jian-hua,Wu Heng,Zhang Wen-bo. Survey on the core techniques of cloud computing[J]. Journal of Chinese Computer Systems,2013,34(11):2417-2424.
[11]Qiu Xin,Gan Chao,Jiang Xiong-xin,et al. Equipment fault diagnosis technology based on apriori algorithm in cloud computing environment[J]. M odular M achine Tool&Automatic M anufacturing Technique,2014,4(12):45-48.
[12]Dong X L,Srivastava D. Big data integration[C]. Data Engineering(ICDE),2013 IEEE 29th International Conference on. IEEE,2013:1245-1248.
[13] Luukka P. Feature selection using fuzzy entropy meas-ures with similarity classifier[J]. Expert Systems w ith Applications,2011,38(4):4600-4607.
[14] Lu Feng,Du Ni,Wen Cheng-lin. A fuzzy-evidential-k nearest neighbor classification algorithm[J]. Acta Electronica Sinica,2012,40(12):2390-2395.
[15]Estévez P A,Tesmer M,Perez C A,et al. Normaliz-ed mutual information feature selection[J]. IEEE Transactions on Neural Netw orks,2009,20(2):189-201.
[16]Guyon I,Elisseeff A. An introduction to variable and feature selection[J]. Journal of M achine Learning Research,2003,3(M ar):1157-1182.
[17] Li Feng,Miao Duo-qian,Zhang Zhi-fei,et al. Mutual information based granular feature w eighted k-nearest neighbors algorithm for multi-label learning[J]. Journal of Computer Research and Development,2017,54(5):1024-1035.
[18]Geng Li-juan,Li Xing-yi. Improvements of KNN alg-orithm for big data classification[J]. Application Research of Computers,2014,31(5):1342-1344.
[19]Chai Sen. Research and design of fault injection platform for cloud computing system[D]. Harbin:Harbin Institute of Technology,2016.
[20] Tan Lan-fang,Tan Qing-ping,Xu Jian-jun. A fault-list generation approach based on data flow analysis[J]. Journal of Chinese Computer Systems,2012,33(4):716-721.
[1]周东华,胡艳艳.动态系统的故障诊断技术[J].自动化学报,2009,35(6):748-758.
[2]王焘,张文博,徐继伟,等.云环境下基于统计监测的分布式软件系统故障检测技术研究[J].计算机学报,2017,40(2):397-413.
[5]付文龙,周建中,李超顺,等.基于模糊K近邻支持向量数据描述的水电机组振动故障诊断研究[J].中国电机工程学报,2014,34(32):5788-5795.
[9]罗军舟,金嘉晖,宋爱波,等.云计算:体系架构与关键技术[J].通信学报,2011,32(7):3-21.
[10]张建华,吴恒,张文博.云计算核心技术研究综述[J].小型微型计算机系统,2013,34(11):2417-2424.
[11]邱昕,甘超,江雄心,等.基于云计算环境下Apriori算法的设备故障诊断技术研究[J].组合机床与自动化加工技术,2014,4(12):45-48.
[17]李峰,苗夺谦,张志飞,等.基于互信息的粒化特征加权多标签学习k近邻算法[J].计算机研究与发展,2017,54(5):1024-1035.
[18]耿丽娟,李星毅.用于大数据分类的KNN算法研究[J].计算机应用研究,2014,31(5):1342-1344.
[19]柴森.云计算系统故障注入平台的研究与设计[D].哈尔滨:哈尔滨工业大学,2016.
[20]谭兰芳,谭庆平,徐建军.一种基于数据流分析的故障序列生成方法[J].小型微型计算机系统,2012,33(4):716-721.
1Darrell Etherington. Gmail And Google+Go Dow n Across The World,Service Returns After Roughly 50 M inutes,http://techcrunch. com/2014/01 /24/gmail-goes-dow n-across-the-w orld,2014.
2 AWS. Summary of the Amazon EC2 and Amazon RDS Service D-isruption in the US East Region,http://aws. amazon. com/cn/message/65648,2011.
[2]Wang Tao,Zhang Wen-bo,Xu Ji-wei,et al. A survey of fault detection for distributed software systems with statistical monitoring in cloud computing[J]. Chinese Journal of Computers,2017,40(2):397-413.
[3] Meng S. Monitoring-as-a-service in the cloud[D]. Georgia Institute of Technology,2012.
[4] Armbrust M,Fox A,Griffith R,et al. A view of cloud computing[J]. Communications of the ACM,2010,53(4):50-58.
[5]Fu Wen-long,Zhou Jian-zhong,Li Chao-shun,et al. Vibrant fault diagnosis for hydro-electric generating unit based on support vector data description improved w ith fuzzy k-nearest neighbor[J]. Proceedings of the CSEE,2014,34(32):5788-5795.
[6]Qing Guan,Zhang Zi-ming,Song Fu. Ensemble of bayesian predictors and decision trees for proactive failure management in cloud computing systems[J]. Journal of Communications,2012,7(1):52-61.
[7]Modi C N,Patel D R,Patel A,et al. Bayesian classifier and snort based netw ork intrusion detection systemin cloud computing[C].Third International Conference on Computing,Communication and Netw orking Technologies(ICCCNT'12),2012:1-7.
[8]Stehle E,Lynch K,Shevertalov M,et al. On the use of computational geometry to detect softw are faults at runtime[C]. Proceedings of the 7th International Conference on Autonomic Computing,ACM,2010:109-118.
[9]Luo Jun-zhou,Jin Jia-hui,Song Ai-bo,et al. Cloud computing:architecture and key technologies[J]. Journal on Communications,2011,32(7):3-21.
[10] Zhang Jian-hua,Wu Heng,Zhang Wen-bo. Survey on the core techniques of cloud computing[J]. Journal of Chinese Computer Systems,2013,34(11):2417-2424.
[11]Qiu Xin,Gan Chao,Jiang Xiong-xin,et al. Equipment fault diagnosis technology based on apriori algorithm in cloud computing environment[J]. M odular M achine Tool&Automatic M anufacturing Technique,2014,4(12):45-48.
[12]Dong X L,Srivastava D. Big data integration[C]. Data Engineering(ICDE),2013 IEEE 29th International Conference on. IEEE,2013:1245-1248.
[13] Luukka P. Feature selection using fuzzy entropy meas-ures with similarity classifier[J]. Expert Systems w ith Applications,2011,38(4):4600-4607.
[14] Lu Feng,Du Ni,Wen Cheng-lin. A fuzzy-evidential-k nearest neighbor classification algorithm[J]. Acta Electronica Sinica,2012,40(12):2390-2395.
[15]Estévez P A,Tesmer M,Perez C A,et al. Normaliz-ed mutual information feature selection[J]. IEEE Transactions on Neural Netw orks,2009,20(2):189-201.
[16]Guyon I,Elisseeff A. An introduction to variable and feature selection[J]. Journal of M achine Learning Research,2003,3(M ar):1157-1182.
[17] Li Feng,Miao Duo-qian,Zhang Zhi-fei,et al. Mutual information based granular feature w eighted k-nearest neighbors algorithm for multi-label learning[J]. Journal of Computer Research and Development,2017,54(5):1024-1035.
[18]Geng Li-juan,Li Xing-yi. Improvements of KNN alg-orithm for big data classification[J]. Application Research of Computers,2014,31(5):1342-1344.
[19]Chai Sen. Research and design of fault injection platform for cloud computing system[D]. Harbin:Harbin Institute of Technology,2016.
[20] Tan Lan-fang,Tan Qing-ping,Xu Jian-jun. A fault-list generation approach based on data flow analysis[J]. Journal of Chinese Computer Systems,2012,33(4):716-721.
[1]周东华,胡艳艳.动态系统的故障诊断技术[J].自动化学报,2009,35(6):748-758.
[2]王焘,张文博,徐继伟,等.云环境下基于统计监测的分布式软件系统故障检测技术研究[J].计算机学报,2017,40(2):397-413.
[5]付文龙,周建中,李超顺,等.基于模糊K近邻支持向量数据描述的水电机组振动故障诊断研究[J].中国电机工程学报,2014,34(32):5788-5795.
[9]罗军舟,金嘉晖,宋爱波,等.云计算:体系架构与关键技术[J].通信学报,2011,32(7):3-21.
[10]张建华,吴恒,张文博.云计算核心技术研究综述[J].小型微型计算机系统,2013,34(11):2417-2424.
[11]邱昕,甘超,江雄心,等.基于云计算环境下Apriori算法的设备故障诊断技术研究[J].组合机床与自动化加工技术,2014,4(12):45-48.
[17]李峰,苗夺谦,张志飞,等.基于互信息的粒化特征加权多标签学习k近邻算法[J].计算机研究与发展,2017,54(5):1024-1035.
[18]耿丽娟,李星毅.用于大数据分类的KNN算法研究[J].计算机应用研究,2014,31(5):1342-1344.
[19]柴森.云计算系统故障注入平台的研究与设计[D].哈尔滨:哈尔滨工业大学,2016.
[20]谭兰芳,谭庆平,徐建军.一种基于数据流分析的故障序列生成方法[J].小型微型计算机系统,2012,33(4):716-721.
1Darrell Etherington. Gmail And Google+Go Dow n Across The World,Service Returns After Roughly 50 M inutes,http://techcrunch. com/2014/01 /24/gmail-goes-dow n-across-the-w orld,2014.
2 AWS. Summary of the Amazon EC2 and Amazon RDS Service D-isruption in the US East Region,http://aws. amazon. com/cn/message/65648,2011.