入侵检测系统的特征提取方法研究及其实现
|
摘要
随着互联网的迅猛发展,黑客攻击日益猖獗,网络安全问题日趋严峻,据统计,全球几乎每20秒就有一起黑客事件发生,仅美国每年由此造成的经济损失就超过100亿美元,因此,网络安全成为当今社会关注的焦点。目前,实现网络安全的技术包括:入侵检测技术、防火墙技术和安全路由器技术等,其中的入侵检测技术由于其对已知的各种入侵具有较好的识别能力,成为P2DR(Policy, Protection, Detection, Response即策略、防护、检测、响应)安全模型的一个重要组成部分,它是动态安全技术的最核心技术之一。
从上世纪80年代Anderson提出入侵检测模型和80年代中期SRI公司设计并成功实现著名的IDES(入侵检测系统)以来,尽管入侵检测技术已取得了很大的发展,但是,随着网络技术的飞速发展与信息传输的实时性要求加快,入侵检测仍面临不少问题,如检测率不高、漏报率较高、检测速度不适应高速网络的发展等。为解决当前入侵检测系统存在的问题,人们正研究在入侵检测中如何采用机器学习方法和数据挖掘技术实现入侵检测的智能化。本文紧密围绕智能入侵检测中特征提取和基于数据挖掘的数据分类这二方面的关键技术开展研究。
1.在入侵检测特征提取方面,研究了采用主成分分析(Principal Component Analysis,PCA)和核主成分分析(Kernel Principal Component Analysis,KPCA)的入侵检测特征提取方法。通过对入侵检测KDDCUP99数据集进行大量对比实验,表明,采用核主成分分析后数据的维数,只有采用主成分分析后数据维数的一半,入侵检测的检测率也提高了近3个百分点。
2.研究了当前智能化入侵检测系统当前存在的问题,并把数据挖掘技术应用于入侵检测系统。论述了基于Apriori算法和CAEP(通过聚集显露模式分类)的入侵特征提取的原理,结合ORACLE9i的数据挖掘引擎,分析其建立分类、预测和关联类的基本数学模型,以及通过JAVA为基础的API来访问这些数学模型的建立和评价(Building and Scoring)功能。
3.设计了一个基于数据挖掘算法并集成神经网络的网络型入侵检测系统的原型。通过在入侵检测系统的引擎中采用数据挖掘算法,不仅提高了系统的检测率,而且,由于采用的数据挖掘算法具有自学习功能,实现了入侵检测的智能化。
With the rapid development of Internet, hackers’attacks are becoming more and more severe, thus Internet security defense is doomed to be a serious concern. It is estimated that a single hacker incident takes place every 20 seconds, just within U.S.A and the total economic loss caused by such attacks amounts to more than one thousand billion U.S dollars in a year. Internet security has been a focus of modern social concern. For the time being, internet security technology includes intrusion detection technology, fire walls, security routers and so on. And among them, intrusion detection systems (IDSs) have relatively better identifying ability against various sorts of intrusions, so IDS turns to be a main part of the P2DR (Policy, Protection, Detection, Response) security model.
From Anderson’s intrusion detection conception model in 1980s and SKI corporation’s designing and successful achievements of the famous IDES to nowadays’IDS products, although intrusion detection technology has made great progresses, it still has some defects and disadvantages such as low detection rate for novel attacks, high frequency of false alarms, etc. To solve this difficult problem in intrusion detection, machine learning and data mining techniques in intelligent IDSs has become a hot topic in the literature. This dissertation focuses on the feature extraction and automatic data classification based on machine learning in intelligent IDSs.
1.Feature extraction methods based on Principal Component Analysis (PCA) and Kernel Principal Component Analysis (KPCA) are studied. Large amounts of experiments for intrusion detection with the KDD-CUP99 dataset are conducted, and the results demonstrate that the data dimension using KPCA is a half of that using PCA and the detection rate of KPCA is improved by 3 percent.
2.Studied the problems in the intelligent intrusion detection,and adopted the data mining in the intrusion detection system. Proposed the principle of Apriori algorithm and CAEP and approve these methods in characteristic extraction,in intrusion detection, unifies ORACLE9i the data mining engine, analyzes its establishment classification, forecast and is connected several kind of basic mathematical models, as well as how to access the building and scoring functions of this model through JAVA-based APIs
3.A system architecture based on data mining and ensemble learning is designed for intelligent intrusion detection systems, so that higher detection rate and learning efficiency can be obtained by using the self-learning function in neural networks.
从上世纪80年代Anderson提出入侵检测模型和80年代中期SRI公司设计并成功实现著名的IDES(入侵检测系统)以来,尽管入侵检测技术已取得了很大的发展,但是,随着网络技术的飞速发展与信息传输的实时性要求加快,入侵检测仍面临不少问题,如检测率不高、漏报率较高、检测速度不适应高速网络的发展等。为解决当前入侵检测系统存在的问题,人们正研究在入侵检测中如何采用机器学习方法和数据挖掘技术实现入侵检测的智能化。本文紧密围绕智能入侵检测中特征提取和基于数据挖掘的数据分类这二方面的关键技术开展研究。
1.在入侵检测特征提取方面,研究了采用主成分分析(Principal Component Analysis,PCA)和核主成分分析(Kernel Principal Component Analysis,KPCA)的入侵检测特征提取方法。通过对入侵检测KDDCUP99数据集进行大量对比实验,表明,采用核主成分分析后数据的维数,只有采用主成分分析后数据维数的一半,入侵检测的检测率也提高了近3个百分点。
2.研究了当前智能化入侵检测系统当前存在的问题,并把数据挖掘技术应用于入侵检测系统。论述了基于Apriori算法和CAEP(通过聚集显露模式分类)的入侵特征提取的原理,结合ORACLE9i的数据挖掘引擎,分析其建立分类、预测和关联类的基本数学模型,以及通过JAVA为基础的API来访问这些数学模型的建立和评价(Building and Scoring)功能。
3.设计了一个基于数据挖掘算法并集成神经网络的网络型入侵检测系统的原型。通过在入侵检测系统的引擎中采用数据挖掘算法,不仅提高了系统的检测率,而且,由于采用的数据挖掘算法具有自学习功能,实现了入侵检测的智能化。
With the rapid development of Internet, hackers’attacks are becoming more and more severe, thus Internet security defense is doomed to be a serious concern. It is estimated that a single hacker incident takes place every 20 seconds, just within U.S.A and the total economic loss caused by such attacks amounts to more than one thousand billion U.S dollars in a year. Internet security has been a focus of modern social concern. For the time being, internet security technology includes intrusion detection technology, fire walls, security routers and so on. And among them, intrusion detection systems (IDSs) have relatively better identifying ability against various sorts of intrusions, so IDS turns to be a main part of the P2DR (Policy, Protection, Detection, Response) security model.
From Anderson’s intrusion detection conception model in 1980s and SKI corporation’s designing and successful achievements of the famous IDES to nowadays’IDS products, although intrusion detection technology has made great progresses, it still has some defects and disadvantages such as low detection rate for novel attacks, high frequency of false alarms, etc. To solve this difficult problem in intrusion detection, machine learning and data mining techniques in intelligent IDSs has become a hot topic in the literature. This dissertation focuses on the feature extraction and automatic data classification based on machine learning in intelligent IDSs.
1.Feature extraction methods based on Principal Component Analysis (PCA) and Kernel Principal Component Analysis (KPCA) are studied. Large amounts of experiments for intrusion detection with the KDD-CUP99 dataset are conducted, and the results demonstrate that the data dimension using KPCA is a half of that using PCA and the detection rate of KPCA is improved by 3 percent.
2.Studied the problems in the intelligent intrusion detection,and adopted the data mining in the intrusion detection system. Proposed the principle of Apriori algorithm and CAEP and approve these methods in characteristic extraction,in intrusion detection, unifies ORACLE9i the data mining engine, analyzes its establishment classification, forecast and is connected several kind of basic mathematical models, as well as how to access the building and scoring functions of this model through JAVA-based APIs
3.A system architecture based on data mining and ensemble learning is designed for intelligent intrusion detection systems, so that higher detection rate and learning efficiency can be obtained by using the self-learning function in neural networks.
引文
[1] Haibin Luo,Yushun Fan.CIMFlow:A Workflow Management System Based on Integration Platform Environment[c].7th IEEE International Conference on Emerging Technologies and Factory Automation, 1999:233-241
[2] 唐正军,李建华.入侵检测技术.北京:清华大学出版社,2004.4
[3] 田丰.基于 P2DR 模型的网络动态安全体系的探讨[J].有线电视技术,2006.6
[4] 中国信息安全产品测评认证中心.信息安全理论与技术.北京:人民邮电出版社,2003.9
[5] 顾巧论,蔡振山,贾春福.计算机网络安全.北京:科学出版社,2003.1
[6] DE Denning.An Intrusion Detection Model. IEEE Transactions on Software Engineering,vol. SE-13, pp. 222-232, February 1990
[7] Heberlein, L. et al. A network Security Monitor. Proceedings of the IEEE Computer Society Symposium, Research in Security and Privacy, May 1990, pp. 296-303
[8] 王璟,史美林等.数据挖掘在协同入侵检测系统中的应用研究.计算机工程与应用,2003,39(21):140-143,146
[9] Amitabh Mishra, Ketan Nadkarni, And Animesh Patcha, Virgina tech.Intrusion Detection In Wireless AD HOC Networks. IEEE Wireless Communications .February 2004,pp.48-60
[10] 王德强, 姜万波等.入侵检测技术研究现状与未来发展.中兴通讯技术, 2004.4
[11] Stephen Yip and David Robson.Appling Formal Specification and Functional Testing Graphical User Interfaces. CompEuro 91. Advanced Computer Technology, Reliable Systems and Applications. 5th Annual European Computer Conference. Proceedings, 13-16 May 2003 Pages:557-561
[12] 郭山清,谢立等.入侵检测在线规则生成模型. 计算机学报, 2006(9):1521-1530
[13] 王晓东 , 石健萍 . 基于事件行为特征的关联分析方法及应用 . 网络信息安全,2006(3)57-58
[14] Koral Ilgun,Richard A Kemmerer,and phillip A Porras. State transition analysis: A rule-based intrusion detection approach. IEEE Transactions on software Engineering, 2004,21(3):181-199
[15] Stuart Staniford Chen.Common intrusion detection framework. http://seclab.cs.ucdavis. edu/cidf/.
[16] IDWG URL:http://www.silicondefense .com/idwg/
[17] Ian T.Jolliffe.Principal Component Analysis.Springer-Verlag,NewYork,2002
[18] Sparse. kernel principal component analysis. In Advances in Neural Information Processing Systems 13. MIT Press.2003
[19] Hiren Shah,Jeffrey Undercoffer and Anupam Joshi.Fuzzy Clustering for Intrusion Detection in Proceedings of the 12th IEEE International Conference on Fuzzy systems,2003.
[20] Susan M. Bridges, Rayford B. Vaughn.Fuzzy Data Mining and Genetic Algorithms Applied to Intrusion Detection. National Informatics Systems Security Conference (NISSC),Baltimore, MD, October 16-19, 2000
[21] 关健,刘大昕.GCIDS:基于遗传分类器的入侵检测与响应系统[J].计算机工程与应用.2003.15,162-166
[22] 肖本贤,昂卫兵,王群京.用混合遗传算法实现神经网络快速训练.合肥工业大学学报(自然科学版).Vol.24 NO.5,Oct 2001, 901-906
[23] 段丹青, 陈松乔等.基于人工免疫的多 Agent 自适应入侵检测系统. 微机发展, 2004:(08)
[24] 吴建鑫,周志华,沈学华,陈兆乾.一种选择性神经网络集成方法,计算机研究与发展,2000,37(9),1039-1044.
[25] 陈荣,高济等.面向网格计算的按需入侵检测模型.浙江大学学报,2006,40(3):387-391
[26] 景志刚,王相林.基于人免疫的网络入检测技术.现代计算机,2005, 2:49-53
[27] Forrest S,Perelson AS,Allen L,et al.Self-Nonself is crimination in a Computer [C].In Proceedings of IEEE Symposium on Research in Security and Privacy,Oakland,May 16-18 1994,202-212
[28] 李辉,管晓宏,昝鑫,韩祟昭,基于支持向量机的网络入侵检测,计算机研究与发展,Vol.40,NO.6 June 2003 799-807
[29] N.Cristionini and J.shawe-Taylor.An Introduction to Support Vector Machines:the web-site associated with the book,2000.
[30] 闫巧,江勇等.基于免疫机理的网络入侵检测系统的抗体生成与检测组件.计算机学报,2005,10:1601-1607
[31] E Osuna,R Freund,F Gtrosi.Training support vector machines.An application to face detection.CVPR'03,Puerto Rico,2003.
[32] Khaled Labib, V. Rao Vemuri.An Application of Principal Component Analysis to theDetection and Visualization of Computer Network Attacks. Annals of Telecommunications, France. Nov/Dec 2005 Issue.
[33] Mei-Ling Shyu.A Novel Anomaly Detection Scheme Based on Principal Component Classifier.Proceedings of ICDM Foundationand New Direction of Data Mining workshop, 2003, 172-179.
[34] 陈莉,刘静等,智能数据挖掘与知识发现.西安:西安电子科技大学出版社,2006:1-5
[35] Wang S H,Wang H.Knowledge Discovery Through self-Organizing Maps:Datat Visualization and Query Processing. Knowledge and Information Systems,2002,4:31-45
[36] [美]Mehmed Kantardzic 著,闪世清,陈茵等(译).数据挖掘—概念、模型、方法和算法[M].北京:清华大学出版社,2003,8:24-30
[37] Wenke Lee.A Data Mining Framework for Constructing Features and Models for Intrusion Detection Systems, PhD thesis ,Columbia A University,2003,6:78-93
[38] [加] Jiawei Han ,Micheline Kamber(著),范明,孟小峰(译).数据挖掘概念与技术[M].北京:机械工业出版社,2003,8:5-12
[39] Markus Hegland . Data Mining – Challenges, Models, Methodsand Algorithms[M].April 7,2004:128-132
[40] 杨 泉 . 数 据 融 合 技 术 在 信 息 安 全 检 测 与 分 析 中 的 应 用 [J]. 网 络 信 息 安全,2005,59(11)53-54
[41] Pang-Ning Tan, Michael Steinbach ,Vipin Kumar(著),范明,范宏建等(译).数据挖掘导论[M].北京:人民邮电出版社,2006.5
[42] Oracle Corporation.Oracle9i Data Mining Concepts Release 9.2.0.2, http ://otn.oracle. com/global/cn/products/bi/9idmining.html,October, 2005
[2] 唐正军,李建华.入侵检测技术.北京:清华大学出版社,2004.4
[3] 田丰.基于 P2DR 模型的网络动态安全体系的探讨[J].有线电视技术,2006.6
[4] 中国信息安全产品测评认证中心.信息安全理论与技术.北京:人民邮电出版社,2003.9
[5] 顾巧论,蔡振山,贾春福.计算机网络安全.北京:科学出版社,2003.1
[6] DE Denning.An Intrusion Detection Model. IEEE Transactions on Software Engineering,vol. SE-13, pp. 222-232, February 1990
[7] Heberlein, L. et al. A network Security Monitor. Proceedings of the IEEE Computer Society Symposium, Research in Security and Privacy, May 1990, pp. 296-303
[8] 王璟,史美林等.数据挖掘在协同入侵检测系统中的应用研究.计算机工程与应用,2003,39(21):140-143,146
[9] Amitabh Mishra, Ketan Nadkarni, And Animesh Patcha, Virgina tech.Intrusion Detection In Wireless AD HOC Networks. IEEE Wireless Communications .February 2004,pp.48-60
[10] 王德强, 姜万波等.入侵检测技术研究现状与未来发展.中兴通讯技术, 2004.4
[11] Stephen Yip and David Robson.Appling Formal Specification and Functional Testing Graphical User Interfaces. CompEuro 91. Advanced Computer Technology, Reliable Systems and Applications. 5th Annual European Computer Conference. Proceedings, 13-16 May 2003 Pages:557-561
[12] 郭山清,谢立等.入侵检测在线规则生成模型. 计算机学报, 2006(9):1521-1530
[13] 王晓东 , 石健萍 . 基于事件行为特征的关联分析方法及应用 . 网络信息安全,2006(3)57-58
[14] Koral Ilgun,Richard A Kemmerer,and phillip A Porras. State transition analysis: A rule-based intrusion detection approach. IEEE Transactions on software Engineering, 2004,21(3):181-199
[15] Stuart Staniford Chen.Common intrusion detection framework. http://seclab.cs.ucdavis. edu/cidf/.
[16] IDWG URL:http://www.silicondefense .com/idwg/
[17] Ian T.Jolliffe.Principal Component Analysis.Springer-Verlag,NewYork,2002
[18] Sparse. kernel principal component analysis. In Advances in Neural Information Processing Systems 13. MIT Press.2003
[19] Hiren Shah,Jeffrey Undercoffer and Anupam Joshi.Fuzzy Clustering for Intrusion Detection in Proceedings of the 12th IEEE International Conference on Fuzzy systems,2003.
[20] Susan M. Bridges, Rayford B. Vaughn.Fuzzy Data Mining and Genetic Algorithms Applied to Intrusion Detection. National Informatics Systems Security Conference (NISSC),Baltimore, MD, October 16-19, 2000
[21] 关健,刘大昕.GCIDS:基于遗传分类器的入侵检测与响应系统[J].计算机工程与应用.2003.15,162-166
[22] 肖本贤,昂卫兵,王群京.用混合遗传算法实现神经网络快速训练.合肥工业大学学报(自然科学版).Vol.24 NO.5,Oct 2001, 901-906
[23] 段丹青, 陈松乔等.基于人工免疫的多 Agent 自适应入侵检测系统. 微机发展, 2004:(08)
[24] 吴建鑫,周志华,沈学华,陈兆乾.一种选择性神经网络集成方法,计算机研究与发展,2000,37(9),1039-1044.
[25] 陈荣,高济等.面向网格计算的按需入侵检测模型.浙江大学学报,2006,40(3):387-391
[26] 景志刚,王相林.基于人免疫的网络入检测技术.现代计算机,2005, 2:49-53
[27] Forrest S,Perelson AS,Allen L,et al.Self-Nonself is crimination in a Computer [C].In Proceedings of IEEE Symposium on Research in Security and Privacy,Oakland,May 16-18 1994,202-212
[28] 李辉,管晓宏,昝鑫,韩祟昭,基于支持向量机的网络入侵检测,计算机研究与发展,Vol.40,NO.6 June 2003 799-807
[29] N.Cristionini and J.shawe-Taylor.An Introduction to Support Vector Machines:the web-site associated with the book,2000.
[30] 闫巧,江勇等.基于免疫机理的网络入侵检测系统的抗体生成与检测组件.计算机学报,2005,10:1601-1607
[31] E Osuna,R Freund,F Gtrosi.Training support vector machines.An application to face detection.CVPR'03,Puerto Rico,2003.
[32] Khaled Labib, V. Rao Vemuri.An Application of Principal Component Analysis to theDetection and Visualization of Computer Network Attacks. Annals of Telecommunications, France. Nov/Dec 2005 Issue.
[33] Mei-Ling Shyu.A Novel Anomaly Detection Scheme Based on Principal Component Classifier.Proceedings of ICDM Foundationand New Direction of Data Mining workshop, 2003, 172-179.
[34] 陈莉,刘静等,智能数据挖掘与知识发现.西安:西安电子科技大学出版社,2006:1-5
[35] Wang S H,Wang H.Knowledge Discovery Through self-Organizing Maps:Datat Visualization and Query Processing. Knowledge and Information Systems,2002,4:31-45
[36] [美]Mehmed Kantardzic 著,闪世清,陈茵等(译).数据挖掘—概念、模型、方法和算法[M].北京:清华大学出版社,2003,8:24-30
[37] Wenke Lee.A Data Mining Framework for Constructing Features and Models for Intrusion Detection Systems, PhD thesis ,Columbia A University,2003,6:78-93
[38] [加] Jiawei Han ,Micheline Kamber(著),范明,孟小峰(译).数据挖掘概念与技术[M].北京:机械工业出版社,2003,8:5-12
[39] Markus Hegland . Data Mining – Challenges, Models, Methodsand Algorithms[M].April 7,2004:128-132
[40] 杨 泉 . 数 据 融 合 技 术 在 信 息 安 全 检 测 与 分 析 中 的 应 用 [J]. 网 络 信 息 安全,2005,59(11)53-54
[41] Pang-Ning Tan, Michael Steinbach ,Vipin Kumar(著),范明,范宏建等(译).数据挖掘导论[M].北京:人民邮电出版社,2006.5
[42] Oracle Corporation.Oracle9i Data Mining Concepts Release 9.2.0.2, http ://otn.oracle. com/global/cn/products/bi/9idmining.html,October, 2005
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |