商业数据仓库系统的设计与实施
|
摘要
如何有效的管理企业在运营过程中产生的大量数据和信息一直是信息系统工作人员面临的重要问题。数据仓库不同于传统运作的事务处理，它们在开发方法、事务处理方法、事务所访问的数据量及数据存储方式等方面有着根本性的区别。本文从数据仓库技术中重要的基本概念入手，详细介绍了在搭建数据仓库过程中几个关键技术：数据抽取、存储管理和数据表现。数据集市同样是数据仓库系统中不可缺少的组成部分。星型模型是数据仓库的数据模型设计一个补充部分，目前数据仓库系统多建在关系数据库之上，所以星型模型是由关系模型模拟多维数据模型的表示和存储，提高了数据仓库访问效率。
数据仓库的系统结构有两种类型：三层结构和两层结构。通过分析这两种结构的优缺点，从而提出了一个新的系统结构：可扩展的系统结构。这种结构可以认为是首先利用了两层结构的较简单和较廉价的优点，同时定义了一种结构路线图。随着需求的不断增加，此线路图允许这种两层结构逐渐的转换为所需要的最为强健的三层结构。明珠数据仓库系统就是采用了可扩展的系统结构，首先从一个主题域——财务主题域入手，而后分别建立了其他三个主题域：部门、商品和配送。
最先建立的财务主题域是个数据集市，它的物理数据库采用了ORACLE关系数据库，而数据集市的数据表现是由POWERBUILDER开发的报表系统。在这部分的论述中，我重点介绍了财务主题域的星型模型建立过程。依据“同等对待多维模型的维和变量”的观点，确定了这个主题域的维结构，同时分析了各个维上独特的层次结构。数据抽取、转换和加载也是重点之一。我详细的分析了在抽取和装载数据到数据仓库过程中所需要解决的问题。我们自行开发的数据表现形式——报表系统，主要特点是应用一个三维结构表现财务主题的多维结构，为财务决策人员提供基于财务报表的数据决策支持。
北京工业大学工学硕士学位论文
一
由于部门、商品和配送三个主题域的加入，使得明珠数据仓库
系统完整的形成了三层数据仓库结构。其中，部门主题域的建立后台
数据库依然是ORACLE，数据分析和表现采用了COGNOS 公司的工
具。
How effective management a large amount of data that enterprise produce in the course of running is a important problem that staff member face. Compare with the transaction processing of traditional operation data warehouse have fundamental differences in development approach, transaction processing method, data amount and data storage that the instance visit. Proceeding with important basic conception in the data warehouse technology,this paper has introduced several pieces of key technology in the course of putting up the data warehouse in detail: the data collect, store managing and data display.The data mart is equally indispensables part in the data warehouse systems.Star type model is one part of supplement that the data model of data warehouse design. At present, the data warehouse systems is more built on the relation database, so the star schema simulates the expression and memory of multidimensional data model by the relation model, for improving visit efficiency.
There are two kinds in the systematic structure of the data warehouse: three tiers of structure and two tiers of structure. Through analyzing the pluses and minuses of these two kinds of structure, the paper put forward a new systematic structure:scalable systematic structure. This kind of structure can think that it utilizes the relatively simple and more low-priced merit of two tiers of structure at first, and defines a kind of structure line map at the same time. With constant increase of demands, these structure line maps allow the two structures for gradual transformation for strong 3 structure most needed. The MingZhu Data Warehouse System adopt the scalable systematic structure. At first starting with the financial theme
land, setting up other 3 theme land respectively: department, goods and providing.
The financial theme land that sets up at first is a data mart, its physics database has adopted ORACLE relation database. And it is systems of report form developed by POWERBUILDER that data display of. the data mart .In the argumentation of this part, I introduce the star schema of financial theme land especially. According to view that "symmetric treatment not only of all dimensions but also of measures", confirm the dimensions of the structure, analyze the unique hierarchical structure on the dimensions at the same time. It is one of the focal points that the data collect, change and data impose. I detailed analyze problem that solve needed in the course of collecting and load the data to the data warehouse jointly. We develop report form system as the form of expression of data. The main characteristic is that use one three-dimensional structure display financial theme multidimensional structure, which offering the decision support of data based on financial statement to financial affairs policy-makers.
Because department, goods and providing three themes join, MingZhu Data Warehouse System form three tiers data warehouse structure. Among them, department theme still adopts the ORACLE database, data analysis and display and adopt COGNOS company tool.
数据仓库的系统结构有两种类型：三层结构和两层结构。通过分析这两种结构的优缺点，从而提出了一个新的系统结构：可扩展的系统结构。这种结构可以认为是首先利用了两层结构的较简单和较廉价的优点，同时定义了一种结构路线图。随着需求的不断增加，此线路图允许这种两层结构逐渐的转换为所需要的最为强健的三层结构。明珠数据仓库系统就是采用了可扩展的系统结构，首先从一个主题域——财务主题域入手，而后分别建立了其他三个主题域：部门、商品和配送。
最先建立的财务主题域是个数据集市，它的物理数据库采用了ORACLE关系数据库，而数据集市的数据表现是由POWERBUILDER开发的报表系统。在这部分的论述中，我重点介绍了财务主题域的星型模型建立过程。依据“同等对待多维模型的维和变量”的观点，确定了这个主题域的维结构，同时分析了各个维上独特的层次结构。数据抽取、转换和加载也是重点之一。我详细的分析了在抽取和装载数据到数据仓库过程中所需要解决的问题。我们自行开发的数据表现形式——报表系统，主要特点是应用一个三维结构表现财务主题的多维结构，为财务决策人员提供基于财务报表的数据决策支持。
北京工业大学工学硕士学位论文
一
由于部门、商品和配送三个主题域的加入，使得明珠数据仓库
系统完整的形成了三层数据仓库结构。其中，部门主题域的建立后台
数据库依然是ORACLE，数据分析和表现采用了COGNOS 公司的工
具。
How effective management a large amount of data that enterprise produce in the course of running is a important problem that staff member face. Compare with the transaction processing of traditional operation data warehouse have fundamental differences in development approach, transaction processing method, data amount and data storage that the instance visit. Proceeding with important basic conception in the data warehouse technology,this paper has introduced several pieces of key technology in the course of putting up the data warehouse in detail: the data collect, store managing and data display.The data mart is equally indispensables part in the data warehouse systems.Star type model is one part of supplement that the data model of data warehouse design. At present, the data warehouse systems is more built on the relation database, so the star schema simulates the expression and memory of multidimensional data model by the relation model, for improving visit efficiency.
There are two kinds in the systematic structure of the data warehouse: three tiers of structure and two tiers of structure. Through analyzing the pluses and minuses of these two kinds of structure, the paper put forward a new systematic structure:scalable systematic structure. This kind of structure can think that it utilizes the relatively simple and more low-priced merit of two tiers of structure at first, and defines a kind of structure line map at the same time. With constant increase of demands, these structure line maps allow the two structures for gradual transformation for strong 3 structure most needed. The MingZhu Data Warehouse System adopt the scalable systematic structure. At first starting with the financial theme
land, setting up other 3 theme land respectively: department, goods and providing.
The financial theme land that sets up at first is a data mart, its physics database has adopted ORACLE relation database. And it is systems of report form developed by POWERBUILDER that data display of. the data mart .In the argumentation of this part, I introduce the star schema of financial theme land especially. According to view that "symmetric treatment not only of all dimensions but also of measures", confirm the dimensions of the structure, analyze the unique hierarchical structure on the dimensions at the same time. It is one of the focal points that the data collect, change and data impose. I detailed analyze problem that solve needed in the course of collecting and load the data to the data warehouse jointly. We develop report form system as the form of expression of data. The main characteristic is that use one three-dimensional structure display financial theme multidimensional structure, which offering the decision support of data based on financial statement to financial affairs policy-makers.
Because department, goods and providing three themes join, MingZhu Data Warehouse System form three tiers data warehouse structure. Among them, department theme still adopts the ORACLE database, data analysis and display and adopt COGNOS company tool.
引文
1 W. H. Inmon Ken Rudin Christopher K. Buss Ryan Sousa. Data Warehouse Performance. John Wiley&Sons, Inc.
2 王珊 等编著．数据仓库技术与联机分析处理．科学出版社1999
3 W. H. Inmon. Building the Data Warehouse, Second Edition. John Wiley&Sons, Inc.
4 杨光，张雷，艾波．数据仓库实施策略．计算机应用．1999年，19：4～7
5 Rakesh Agrawal, Ashish Gupta, Sunita Sarawagi. Modeling Multidimensional Databases. Jackson M, Pu C eds. Proceedings of the 13th International Conference on Data Engineering, Los Alamitos, CA, 1997. IEEE Society Press, 1997:105～116
6 孙林，张培衢．财务数据仓库系统的分析设计与实现．河南师范大学学报(自然科学版)．2000，28(4)：88～91
7 方男晖，朱斌，张为．多维数据库分析系统在南航财务分析中的应用．暨南大学学报(自然科学版)．2001，22(5)：69～73
8 孙林，张培衢，高亚萍．企业财务管理辅助决策探讨．计算机应用．2001，21(1)：89～90
9 连立贵，金风，蔡家楣．数据仓库中的数据提取．计算机工程．2001，27(9)：61～63
10 李竹平，吴香林．基于数据仓库的企业管理型会计信息系统研究．华中理工大学学报．2000，28(1)：10～12
11 马洪杰，曲晓飞，刘波．数据仓库系统的三层开发结构及其应用．大连理工大学学报．2000，第40卷：
12 马宏鹏，赵新，李明，卢桂章．数据仓库原型系统设计．计算机工程与应用．2000，(11)：109～112
13 陈俊杰．数据仓库与决策支持系统．太原理工大学学报．1999，30(11)：
14 史金红，吴永明．数据仓库中元数据的管理．计算机应用．2000，(2)：9～12
15 袁磊，陈长清，冯玉才，数据仓库的信息供应链模型．计算机工程与应用．2001， (22)：121～131
16 廖磷，王立刚，刘文煜．构造数据仓库系统的元数据．计算机工程与应用．2001，(16)：94～96
17 吕波，王延章，王红梅．数据仓库元数据的界定与分类．信息与控制．2001，30(6)：498～507
18 张伟民，张小英，居娣．数据仓库技术在电信管理网中的应用．计算机应用．2000，(2)：5～9
19 李杰．数据仓库系统在商业连锁企业中的应用．北京工业大学计算机学院硕士论文．2001
20 Chang Li, X. Sean Wang. A Data Model For Support On-line Analytical Processing. Barker K, Manitoba U eds. Proceedings of the 5th International Conference on Information and Knowledge Management, New York, 1996. Springer-Verlag 1996:81～88
21 Gyssens M, Lakshmansn L.V.S.A Foundation for Multi-Dimensional Databases. Dayal U, Gray P M D, Nishio S eds. Proceedings of the 23rd VLDB Conference on very large data bases, San Francisco, CA: Morgan Kaufmann Publishers, Inc., 1997. 106～115
22 Lehner W. Modeling large scale OLAP scenarios. Jorg H, Ramos Ieds. Proceedings of the 6th International Conference on Extending Database Technology, New York, 1998. Springer-Verlag, 1998: 153～167
23 Gray J, Bosworth A, Layman A et al. Data cube: a relational aggregation operator generalizing group-by, cross-tab and sub-totals. Data Mining and Knowledge Discovery. 1997, 1(1) : 29～54
24 Datta A, Thomas H.A conceptual model and algebra for on-line analytical processing in decision support databases. Thomas H eds. Proceedings of the 7th Annual Workshop on Information Technologies and Systems, San Mateo, CA, 1997. Morgan Kaufman Publishers, Inc, 1997:91～100
25 Pedersen T.B, Jensen C.S. Multidimensional data modeling for complex data. Kitsuregawa M, Maciaszek L, Papazoglou M et al eds. Proceedings of the 15th International Conference on Data Engineering, Los Alamitos, CA, 1999. IEEE Society Press, 1999:336～345
26 Coliat G. OLAP, relational, and multidimensional database system. SIGMOD Record. 1996,25(3):64～69
27 Codd E F. Providing OLAP (on-line analytical processing) to user-analysts: an IT mandate. Technical Report, 1993
28 Rafanelli M, Shoshani A. STORM. a statistical object representation model. Data Engineering Bulletion, 1990,13(3):12～18
29 汪永生，邵惠鹤．CIPS 中的数据仓库技术．计算机应用．2000，27(1)：36～40
30 邓文华，梅志红．基于数据仓库的铁路管理决策支持系统的研究．华东交通大学学报．2000，16(2)：31～34
31 张海英，张斌，王云，刘杰，范春晓．数据仓库技术在SDH配置管理中的应用．2000，23(2)：77～81
32 李子目，徐明，周兴铭．数据仓库的集合刷新维护算法．计算机工程与科学．2000，22(5)：81～85
33 Joseph M. Firestone. Object-Oriented Data Warehousing. Executive Information Systems, Inc. 1997
34 Joseph M. Firestone. Dimensional Modeling and E-R Modeling In The Data Warehouse. Executive Information Systems, Inc. 1997
35 Marc Demarest. Building The Data Mart. DBMS Magazine. 1993, (10)
36 Bohn, Kathy. Coverting Data for Warehouse. DBMS Magazine. 1997, (6)
37 Hufford Duane. Data Warehouse Quality. Data Management Review. 1996, (3)
38 Chanuduri, Surajit, Dayal, Umenshwar. An Overview of Data Warehouse and OLAP Technology. ACM Sigmod Record. 1997, (3)
39 Rich Carickhoff. A New Face For OLAP. Internet Systems. 1997, (1)
40 Nigel Pendse. What is OLAP. Business Intelligence Ltd. 2000
2 王珊 等编著．数据仓库技术与联机分析处理．科学出版社1999
3 W. H. Inmon. Building the Data Warehouse, Second Edition. John Wiley&Sons, Inc.
4 杨光，张雷，艾波．数据仓库实施策略．计算机应用．1999年，19：4～7
5 Rakesh Agrawal, Ashish Gupta, Sunita Sarawagi. Modeling Multidimensional Databases. Jackson M, Pu C eds. Proceedings of the 13th International Conference on Data Engineering, Los Alamitos, CA, 1997. IEEE Society Press, 1997:105～116
6 孙林，张培衢．财务数据仓库系统的分析设计与实现．河南师范大学学报(自然科学版)．2000，28(4)：88～91
7 方男晖，朱斌，张为．多维数据库分析系统在南航财务分析中的应用．暨南大学学报(自然科学版)．2001，22(5)：69～73
8 孙林，张培衢，高亚萍．企业财务管理辅助决策探讨．计算机应用．2001，21(1)：89～90
9 连立贵，金风，蔡家楣．数据仓库中的数据提取．计算机工程．2001，27(9)：61～63
10 李竹平，吴香林．基于数据仓库的企业管理型会计信息系统研究．华中理工大学学报．2000，28(1)：10～12
11 马洪杰，曲晓飞，刘波．数据仓库系统的三层开发结构及其应用．大连理工大学学报．2000，第40卷：
12 马宏鹏，赵新，李明，卢桂章．数据仓库原型系统设计．计算机工程与应用．2000，(11)：109～112
13 陈俊杰．数据仓库与决策支持系统．太原理工大学学报．1999，30(11)：
14 史金红，吴永明．数据仓库中元数据的管理．计算机应用．2000，(2)：9～12
15 袁磊，陈长清，冯玉才，数据仓库的信息供应链模型．计算机工程与应用．2001， (22)：121～131
16 廖磷，王立刚，刘文煜．构造数据仓库系统的元数据．计算机工程与应用．2001，(16)：94～96
17 吕波，王延章，王红梅．数据仓库元数据的界定与分类．信息与控制．2001，30(6)：498～507
18 张伟民，张小英，居娣．数据仓库技术在电信管理网中的应用．计算机应用．2000，(2)：5～9
19 李杰．数据仓库系统在商业连锁企业中的应用．北京工业大学计算机学院硕士论文．2001
20 Chang Li, X. Sean Wang. A Data Model For Support On-line Analytical Processing. Barker K, Manitoba U eds. Proceedings of the 5th International Conference on Information and Knowledge Management, New York, 1996. Springer-Verlag 1996:81～88
21 Gyssens M, Lakshmansn L.V.S.A Foundation for Multi-Dimensional Databases. Dayal U, Gray P M D, Nishio S eds. Proceedings of the 23rd VLDB Conference on very large data bases, San Francisco, CA: Morgan Kaufmann Publishers, Inc., 1997. 106～115
22 Lehner W. Modeling large scale OLAP scenarios. Jorg H, Ramos Ieds. Proceedings of the 6th International Conference on Extending Database Technology, New York, 1998. Springer-Verlag, 1998: 153～167
23 Gray J, Bosworth A, Layman A et al. Data cube: a relational aggregation operator generalizing group-by, cross-tab and sub-totals. Data Mining and Knowledge Discovery. 1997, 1(1) : 29～54
24 Datta A, Thomas H.A conceptual model and algebra for on-line analytical processing in decision support databases. Thomas H eds. Proceedings of the 7th Annual Workshop on Information Technologies and Systems, San Mateo, CA, 1997. Morgan Kaufman Publishers, Inc, 1997:91～100
25 Pedersen T.B, Jensen C.S. Multidimensional data modeling for complex data. Kitsuregawa M, Maciaszek L, Papazoglou M et al eds. Proceedings of the 15th International Conference on Data Engineering, Los Alamitos, CA, 1999. IEEE Society Press, 1999:336～345
26 Coliat G. OLAP, relational, and multidimensional database system. SIGMOD Record. 1996,25(3):64～69
27 Codd E F. Providing OLAP (on-line analytical processing) to user-analysts: an IT mandate. Technical Report, 1993
28 Rafanelli M, Shoshani A. STORM. a statistical object representation model. Data Engineering Bulletion, 1990,13(3):12～18
29 汪永生，邵惠鹤．CIPS 中的数据仓库技术．计算机应用．2000，27(1)：36～40
30 邓文华，梅志红．基于数据仓库的铁路管理决策支持系统的研究．华东交通大学学报．2000，16(2)：31～34
31 张海英，张斌，王云，刘杰，范春晓．数据仓库技术在SDH配置管理中的应用．2000，23(2)：77～81
32 李子目，徐明，周兴铭．数据仓库的集合刷新维护算法．计算机工程与科学．2000，22(5)：81～85
33 Joseph M. Firestone. Object-Oriented Data Warehousing. Executive Information Systems, Inc. 1997
34 Joseph M. Firestone. Dimensional Modeling and E-R Modeling In The Data Warehouse. Executive Information Systems, Inc. 1997
35 Marc Demarest. Building The Data Mart. DBMS Magazine. 1993, (10)
36 Bohn, Kathy. Coverting Data for Warehouse. DBMS Magazine. 1997, (6)
37 Hufford Duane. Data Warehouse Quality. Data Management Review. 1996, (3)
38 Chanuduri, Surajit, Dayal, Umenshwar. An Overview of Data Warehouse and OLAP Technology. ACM Sigmod Record. 1997, (3)
39 Rich Carickhoff. A New Face For OLAP. Internet Systems. 1997, (1)
40 Nigel Pendse. What is OLAP. Business Intelligence Ltd. 2000
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |