基于点击流分析的电子商务智能决策支持系统
|
摘要
随着基于因特网的各种信息系统在企业中的应用，企业将收集越来越多的关于客户、产品及销售情况在内的各种信息，这些信息能帮助企业更好地预测和把握未来。在决策支持系统基础上进一步发展起来的商务智能能够向用户提供更为复杂的商业信息，使分析处理信息的能力和信息的利用率大为提高，从而进一步解决企业决策时需要了解的各种问题，并帮助企业更快更好地制定和做出决策。由此可见，在决策支持系统基础上发展商务智能已成为现代企业发展的势不可挡的必然趋势。
本文在综述决策支持系统发展及目前研究现状的基础上，提出了以客户的点击流数据为基础，并且将点击流数据和商务网站后台数据相结合，共同进行智能化分析，以实现一个电子商务环境下的智能决策支持系统设计与开发的研究思想。在进行智能化分析时，我们引入了数据挖掘的技术。
论文首先给出了系统的整体框架体系结构设计，以及主要的功能模块介绍；接着，在数据预处理部分，设计了在应用层收集点击流数据并且对其进行实时预处理的方法；在数据挖掘即数据分析部分，研究与实现了用于数据概化分析的面向属性规约的扩展算法，以及设计并实现了用于关连分析的混合维关联规则挖掘算法；最后，在江苏长江电气集团的电子商务网站系统上，利用已分析的算法设计并实现了一个智能决策支持系统。
论文的创新之处在于：
1．提出基于点击流挖掘技术的商务智能辅助决策系统框架。从大量的客户访问数据中挖掘得出能指导企业生产、辅助企业商务智能决策的有用信息。
2．对单维、单层、布尔型关联规则挖掘算法——Apriori算法进行改进，设计并实现了基于多维事务数据库的混合维关联规则挖掘算法。
3．利用构件化的思想设计挖掘算法库，实现了算法库的可扩展性和易选择性。
With the application of all kinds of information systems based on Internet, all corporations would collect more information concerning customers, productions and sales. And those information could help corporations to forecast and grasp the future situation better. The Business Intelligence System, which depends on Decision Support System (DSS), could provide the users more complex business information, and greatly improve the capability of analyzing and handling information, together with the utility of information, so that helps to solve many problems in the decision-making and make the project more promptly and more perfectly. Therefore, developing Business Intelligence System on the basis of DSS has become the inundant trend of developing modern corporations.
On the survey of the development and current researching statement of DSS, this paper present such a research idea: analyzing the customers' Click-stream data and inner data of corporations intelligently, so that fulfilling the design and development of an Intelligent DSS in the environment of e_business. The technology of Data Mining is introduced during intelligent analysis.
In this paper, we firstly present the whole framework of the system, including the introduction of the main functional module. Next, in the part of data preprocessing, we design a method of collecting click-stream data in the application server layer and preprocessing them with real time; In the part of Data Mining that is data analyzing, we research and implement an extended attribute-oriented induction algorithm which applies to data Generalization analysis, and that, we also design and implement an hybrid-dimensional association rule mining algorithm for associative analysis. In the end, on the e-business web site system of Jiangsu Changjiang electronic Group Corp, we design and implement an Intelligent DSS(IDSS) with the help of the above algorithms.
The innovations of this paper rest with the following:
1. Present a system framework of intelligent DSS based on click-stream data mining technology. Much of useful information, which could guide the production and assist decision making in corporations, could be mined from the enormous customers' visiting data.
2. Improve the Apriori algorithm for mining the single-dimensional, single-level, Boolean association rules. Design and implement the algorithm which can mining hybrid -dimensional association rules within multi-dimensional transaction database.
3. Implement the expansibility and selectivity of algorithms library, in term of the ideal of component
本文在综述决策支持系统发展及目前研究现状的基础上，提出了以客户的点击流数据为基础，并且将点击流数据和商务网站后台数据相结合，共同进行智能化分析，以实现一个电子商务环境下的智能决策支持系统设计与开发的研究思想。在进行智能化分析时，我们引入了数据挖掘的技术。
论文首先给出了系统的整体框架体系结构设计，以及主要的功能模块介绍；接着，在数据预处理部分，设计了在应用层收集点击流数据并且对其进行实时预处理的方法；在数据挖掘即数据分析部分，研究与实现了用于数据概化分析的面向属性规约的扩展算法，以及设计并实现了用于关连分析的混合维关联规则挖掘算法；最后，在江苏长江电气集团的电子商务网站系统上，利用已分析的算法设计并实现了一个智能决策支持系统。
论文的创新之处在于：
1．提出基于点击流挖掘技术的商务智能辅助决策系统框架。从大量的客户访问数据中挖掘得出能指导企业生产、辅助企业商务智能决策的有用信息。
2．对单维、单层、布尔型关联规则挖掘算法——Apriori算法进行改进，设计并实现了基于多维事务数据库的混合维关联规则挖掘算法。
3．利用构件化的思想设计挖掘算法库，实现了算法库的可扩展性和易选择性。
With the application of all kinds of information systems based on Internet, all corporations would collect more information concerning customers, productions and sales. And those information could help corporations to forecast and grasp the future situation better. The Business Intelligence System, which depends on Decision Support System (DSS), could provide the users more complex business information, and greatly improve the capability of analyzing and handling information, together with the utility of information, so that helps to solve many problems in the decision-making and make the project more promptly and more perfectly. Therefore, developing Business Intelligence System on the basis of DSS has become the inundant trend of developing modern corporations.
On the survey of the development and current researching statement of DSS, this paper present such a research idea: analyzing the customers' Click-stream data and inner data of corporations intelligently, so that fulfilling the design and development of an Intelligent DSS in the environment of e_business. The technology of Data Mining is introduced during intelligent analysis.
In this paper, we firstly present the whole framework of the system, including the introduction of the main functional module. Next, in the part of data preprocessing, we design a method of collecting click-stream data in the application server layer and preprocessing them with real time; In the part of Data Mining that is data analyzing, we research and implement an extended attribute-oriented induction algorithm which applies to data Generalization analysis, and that, we also design and implement an hybrid-dimensional association rule mining algorithm for associative analysis. In the end, on the e-business web site system of Jiangsu Changjiang electronic Group Corp, we design and implement an Intelligent DSS(IDSS) with the help of the above algorithms.
The innovations of this paper rest with the following:
1. Present a system framework of intelligent DSS based on click-stream data mining technology. Much of useful information, which could guide the production and assist decision making in corporations, could be mined from the enormous customers' visiting data.
2. Improve the Apriori algorithm for mining the single-dimensional, single-level, Boolean association rules. Design and implement the algorithm which can mining hybrid -dimensional association rules within multi-dimensional transaction database.
3. Implement the expansibility and selectivity of algorithms library, in term of the ideal of component
引文
[1] Gorry G A, Scott Morton M S. A Framework for Management Information Systems. Sloan Management Review, 1971.
[2] 陈文伟．决策支持系统及其开发[M]．北京出版社，1994．
[3] 谢康林．智能化决策支持系统结构框架的研究[J]．上海交通大学学报，1996，30(6)：76～80．
[4] Zai99 [ZAIANE-92]O.Zaiane Resource and Knowledge Discovery from the Internet and Multimedia Repositories. 1999. Ph.DThesis.
[5] Cooley R, Mobasher B, Srivastava J. Web mining: Information and pattern discovery on the World Wide Web. In: Proc of the 9th Int' l Conf on Tools with Artificial Intelligence(ICTA'97). CA, 1997. 558-567.
[6] 郑宏珍，柳明欣，数据挖掘及其工具的使用．
[7] Jiawei Han，Michelme Kamber著，范明，孟小峰等译．数据挖掘概念与技术．机械工业出版社．
[8] Nasraoui O, Frigui H, Joshi A et al. Mining Web access logs using relational competitive fuzzy clustering[C].
[9] 《Oracle9i for e-Business: Business Intelligence》,An Oracle Technical White Paper, June 2001.
[10] http://www. w3. org/pub/WWW
[11] Bamshad Mobasher R. Coolery and J. Srivastava, Data Preparation for Mining World Wide Web Browsing Patterns, Journal of Knowledge and Information Systems, Vol. 1, No. 1,1999.
[12] Bamshad Mobasher, R. Cooley and J. Srivastava, Gouping Web Page Rferences into Transactions for Mining World Wide Web Browsing Patterns, Proceedings of the 1997 IEEE Knowledge and Data Enginering Exchange Workshop (KDEX-97), November 1997.
[13] Suhail Ansari, Ron Kohavi, Llew Mason, and Zijian Zheng. Integrating E-Commerce and
Data Mining: Architecture and Challenges. Appeared in ICDM'01: The 2001 IEEE International Conference on Data Mining. Avaiable at http://www.kohavi.com.
[14] Ron Kohavi, Mining E-Commerce Data: The Good, the Bad, and the Ugly. To appear as invited paper at KDD 2001's Industrial Track.
[15] 张娥，《Web用户访问日志数据挖掘方法研究与实践》，西安交通大学硕士学位论文，2001．
[16] 刘同明 等编著，《数据挖掘技术及其应用》，国防工业出版社，2001．
[17] Han J, Cai Y and Cercone N. Knowledge Discovery in Databases: An Attribute-Oriented Approach[OL].
[18] R. Agrawal, T. Imielinski, and A. Swami. Mining association rules between sets of items in large databases. Proceedings of the ACM SIGMOD Conference on Management of data, pp. 207-216, 1993.
[19] H. Mannila, H. Toivonen, and A. Verkamo. Efficient algorithm for discovering association rules. AAAI Workshop on Knowledge Discovery in Databases, 1994.
[20] Jiawei Han, Jian Pei, and Yiwen Yin, Mining Frequent Patterns without Candidate Generation, in Proc. Of the ACM Int. Conf. On Management od Data, Dallas, TX, May 2000.
[21] Maurice Houtsma, A run Swam i. Set-oriented mining of association rules[A]. Research Report RJ 9567, IBM Almaden Research Center[C]. San Jose, California:[s.n.] 1993.
[22] A. Savasere, E. Omiecinski, and S. Navathe. An efficient algorithm for mining association rules in large databases. Proceedings of the 21st International Conference on Very large Database, 1995.
[23] 欧阳为民，郑诚，蔡庆生．国际上关联规则发现研究述评．计算机科学，1999,26(3)：41-45
[24] Myra Spiliopoulou abd Lukas C. Faulstich, WUM: a Web Utilization Miner, In Workshop on the Web and Data Bases(WebDB98), Page 109-115,1998.
[25] T.Joachim, D. Freitag, and T. Mitchell. Webwatcher: A tour guide for the world wide web. In the 15th Inter-national Conference on Artificial Intelligence, Nagoya, Japan, 1997.
[261 Robert Coolery, Pang-Ning Tan, Jaideep Srivastava, WebSIFT: The Web Site Information Filter System, Proceeding of the Web Usage Analysis and User Profiling Workshop, August 1999.
[27] Mike Perkowitz and Oren Etzioni, Adaptive Web Sites: Automatically Synthesizing web Pages, In Proceedings of Fifteenth National Conference on Artificial Intelligence, Madison, WI, 1997.
[28] Information Discovery, Inc.OLAP and DataMining, Bridging the Gap.
[29] O. R. Zaane, M. Xin, and J. Han. Discovering Web Access Patterns and Trends by Applying OLAP and Data Mining Technology on Web Logs. In Proceedings of the Fifth IEEE Forum on Research and Technology Advances in Digital Libraries, pp. 19-29,1998.
[30] K.P. Joshi, A. Joshi, Y. Yesha, and R. Krishnapuram. Warehousing and Mining Web Logs. In Proceedings of the Second ACM Workshop on Web Information and Data Management, pp. 63-68, 1999.
[31] A. Joshi and R. Krishnapuram. On Mining Web Access Logs. In Proceedings of the 2000 ACM SIGMOD Workshop on Research Issues in Data Mining and Knowledge Discovery 2000, pp. 63-69, 2000.
[2] 陈文伟．决策支持系统及其开发[M]．北京出版社，1994．
[3] 谢康林．智能化决策支持系统结构框架的研究[J]．上海交通大学学报，1996，30(6)：76～80．
[4] Zai99 [ZAIANE-92]O.Zaiane Resource and Knowledge Discovery from the Internet and Multimedia Repositories. 1999. Ph.DThesis.
[5] Cooley R, Mobasher B, Srivastava J. Web mining: Information and pattern discovery on the World Wide Web. In: Proc of the 9th Int' l Conf on Tools with Artificial Intelligence(ICTA'97). CA, 1997. 558-567.
[6] 郑宏珍，柳明欣，数据挖掘及其工具的使用．
[7] Jiawei Han，Michelme Kamber著，范明，孟小峰等译．数据挖掘概念与技术．机械工业出版社．
[8] Nasraoui O, Frigui H, Joshi A et al. Mining Web access logs using relational competitive fuzzy clustering[C].
[9] 《Oracle9i for e-Business: Business Intelligence》,An Oracle Technical White Paper, June 2001.
[10] http://www. w3. org/pub/WWW
[11] Bamshad Mobasher R. Coolery and J. Srivastava, Data Preparation for Mining World Wide Web Browsing Patterns, Journal of Knowledge and Information Systems, Vol. 1, No. 1,1999.
[12] Bamshad Mobasher, R. Cooley and J. Srivastava, Gouping Web Page Rferences into Transactions for Mining World Wide Web Browsing Patterns, Proceedings of the 1997 IEEE Knowledge and Data Enginering Exchange Workshop (KDEX-97), November 1997.
[13] Suhail Ansari, Ron Kohavi, Llew Mason, and Zijian Zheng. Integrating E-Commerce and
Data Mining: Architecture and Challenges. Appeared in ICDM'01: The 2001 IEEE International Conference on Data Mining. Avaiable at http://www.kohavi.com.
[14] Ron Kohavi, Mining E-Commerce Data: The Good, the Bad, and the Ugly. To appear as invited paper at KDD 2001's Industrial Track.
[15] 张娥，《Web用户访问日志数据挖掘方法研究与实践》，西安交通大学硕士学位论文，2001．
[16] 刘同明 等编著，《数据挖掘技术及其应用》，国防工业出版社，2001．
[17] Han J, Cai Y and Cercone N. Knowledge Discovery in Databases: An Attribute-Oriented Approach[OL].
[18] R. Agrawal, T. Imielinski, and A. Swami. Mining association rules between sets of items in large databases. Proceedings of the ACM SIGMOD Conference on Management of data, pp. 207-216, 1993.
[19] H. Mannila, H. Toivonen, and A. Verkamo. Efficient algorithm for discovering association rules. AAAI Workshop on Knowledge Discovery in Databases, 1994.
[20] Jiawei Han, Jian Pei, and Yiwen Yin, Mining Frequent Patterns without Candidate Generation, in Proc. Of the ACM Int. Conf. On Management od Data, Dallas, TX, May 2000.
[21] Maurice Houtsma, A run Swam i. Set-oriented mining of association rules[A]. Research Report RJ 9567, IBM Almaden Research Center[C]. San Jose, California:[s.n.] 1993.
[22] A. Savasere, E. Omiecinski, and S. Navathe. An efficient algorithm for mining association rules in large databases. Proceedings of the 21st International Conference on Very large Database, 1995.
[23] 欧阳为民，郑诚，蔡庆生．国际上关联规则发现研究述评．计算机科学，1999,26(3)：41-45
[24] Myra Spiliopoulou abd Lukas C. Faulstich, WUM: a Web Utilization Miner, In Workshop on the Web and Data Bases(WebDB98), Page 109-115,1998.
[25] T.Joachim, D. Freitag, and T. Mitchell. Webwatcher: A tour guide for the world wide web. In the 15th Inter-national Conference on Artificial Intelligence, Nagoya, Japan, 1997.
[261 Robert Coolery, Pang-Ning Tan, Jaideep Srivastava, WebSIFT: The Web Site Information Filter System, Proceeding of the Web Usage Analysis and User Profiling Workshop, August 1999.
[27] Mike Perkowitz and Oren Etzioni, Adaptive Web Sites: Automatically Synthesizing web Pages, In Proceedings of Fifteenth National Conference on Artificial Intelligence, Madison, WI, 1997.
[28] Information Discovery, Inc.OLAP and DataMining, Bridging the Gap.
[29] O. R. Zaane, M. Xin, and J. Han. Discovering Web Access Patterns and Trends by Applying OLAP and Data Mining Technology on Web Logs. In Proceedings of the Fifth IEEE Forum on Research and Technology Advances in Digital Libraries, pp. 19-29,1998.
[30] K.P. Joshi, A. Joshi, Y. Yesha, and R. Krishnapuram. Warehousing and Mining Web Logs. In Proceedings of the Second ACM Workshop on Web Information and Data Management, pp. 63-68, 1999.
[31] A. Joshi and R. Krishnapuram. On Mining Web Access Logs. In Proceedings of the 2000 ACM SIGMOD Workshop on Research Issues in Data Mining and Knowledge Discovery 2000, pp. 63-69, 2000.