综合录井仪数据迁移及转换接口技术研究
|
摘要
目前综合录井数据大部分保存在综合录井仪自身配带的Access、SQL Server等数据库中,数据格式不统一,数据项不全面,数据相互之间不能共享,使的这些宝贵的数据得不到充分的利用。因此,结合目前已有的数据迁移转换技术,设计一个综合录井仪数据迁移转换接口是十分必要的,将目前来源离散且格式多样的录井数据进行标准化处理,再将这些经过标准化处理的规范数据迁移到一个综合数据库中,为给研究部门和管理部门提供标准确的数据信息,实现数据共享打基础。
本接口的设计是基于中间件技术和XML(eXtensible Markup Language,可扩展标记语言)技术的,采用中间件技术实现数据迁移,不仅能屏蔽掉底层综合录井仪的异构性,实现用户对数据库的透明操作,还可以使接口具有良好的可扩展性。采用XML技术进行数据转换,不仅因为XML的开放性、可扩展性及XSLT(eXtensible Stylesheet Language for Transformations,扩展样式转换语言)对数据格式转换功能,是转换油田复杂数据源所需要的特点,更重要的是因为WITSML(Wellsite Information Transfer Standard Markup Language)是基于XML的井场信息传输标准标记语言,并且日益成为钻井数据传输和保存的标准语言。
本接口由三个相对独立且相互关联的部分组成,即数据查询模块、数据转换模块和数据加载模块。其中数据查询模块负责实现用户对各录井仪异构数据库的透明访问,选择需要迁移的数据库内容;数据转换负责将选中的需迁移数据从数据库中抽取出来,并将其转换成统一的XML文档呈现给用户;数据加载模块负责将XML文档中的信息加载到目标数据库中去。最后,通过实现Access数据库和SQL Server数据库中的数据到Oracle数据库的迁移,验证了数据迁移转换接口方案的可行性。
Recently, most comprehensive logging data are stored in their own databases such as Access, SQL Server and so on. The data formats of these logging data are not unified and the data can not be shared with each other, so they are not utilized fully. It is very necessary to design a comprehensive logging Data Migration Interface, which can combine the currently available data migration and conversion technology. By this interface, we can standardize the comprehensive logging data firstly which are discrete in sources and variety in formats, and then transfer these standardization datum into an integrated database. The purpose of interface designing is to provide the standard data for the research department and management, and lay the foundation for data sharing.
The interface design is based on middleware technology and XML (eXtensible Markup Language) technology. We use middleware technology to realize data transfer. This technology can not only suppress complexity and heterogeneity of comprehensive logging, but also make interface a good scalability. The use of XML technology to realize data transform have two reasons. The first is because XML has many features such as opening, expansibility, data conversation function by XSLT and so on, which is precisely required by oil complex data. The second reason is more important, because WITSML is an XML-based market information transmission Well Standard markup language, and it is increasingly becoming the standard language on drilling data transmission and preservation.
The interface is made up of three both independent and interrelated components. There are data query module, data conversion module and data load module. Data query module will be responsible for the realization that logging instrument databases are visited by users transparently and select database which content need to be transfered. Data conversion module will be responsible for extracting data content from database which are selected, and then transforming them into a unified XML documents which will be presented to the user. Data load modules will be responsible for loading the data which are in XML documents to the target database. In the last section of paper, the feasibility of the interface is proved through an examination that realizing data migration from Access and SQL Server database to Oracle database.
本接口的设计是基于中间件技术和XML(eXtensible Markup Language,可扩展标记语言)技术的,采用中间件技术实现数据迁移,不仅能屏蔽掉底层综合录井仪的异构性,实现用户对数据库的透明操作,还可以使接口具有良好的可扩展性。采用XML技术进行数据转换,不仅因为XML的开放性、可扩展性及XSLT(eXtensible Stylesheet Language for Transformations,扩展样式转换语言)对数据格式转换功能,是转换油田复杂数据源所需要的特点,更重要的是因为WITSML(Wellsite Information Transfer Standard Markup Language)是基于XML的井场信息传输标准标记语言,并且日益成为钻井数据传输和保存的标准语言。
本接口由三个相对独立且相互关联的部分组成,即数据查询模块、数据转换模块和数据加载模块。其中数据查询模块负责实现用户对各录井仪异构数据库的透明访问,选择需要迁移的数据库内容;数据转换负责将选中的需迁移数据从数据库中抽取出来,并将其转换成统一的XML文档呈现给用户;数据加载模块负责将XML文档中的信息加载到目标数据库中去。最后,通过实现Access数据库和SQL Server数据库中的数据到Oracle数据库的迁移,验证了数据迁移转换接口方案的可行性。
Recently, most comprehensive logging data are stored in their own databases such as Access, SQL Server and so on. The data formats of these logging data are not unified and the data can not be shared with each other, so they are not utilized fully. It is very necessary to design a comprehensive logging Data Migration Interface, which can combine the currently available data migration and conversion technology. By this interface, we can standardize the comprehensive logging data firstly which are discrete in sources and variety in formats, and then transfer these standardization datum into an integrated database. The purpose of interface designing is to provide the standard data for the research department and management, and lay the foundation for data sharing.
The interface design is based on middleware technology and XML (eXtensible Markup Language) technology. We use middleware technology to realize data transfer. This technology can not only suppress complexity and heterogeneity of comprehensive logging, but also make interface a good scalability. The use of XML technology to realize data transform have two reasons. The first is because XML has many features such as opening, expansibility, data conversation function by XSLT and so on, which is precisely required by oil complex data. The second reason is more important, because WITSML is an XML-based market information transmission Well Standard markup language, and it is increasingly becoming the standard language on drilling data transmission and preservation.
The interface is made up of three both independent and interrelated components. There are data query module, data conversion module and data load module. Data query module will be responsible for the realization that logging instrument databases are visited by users transparently and select database which content need to be transfered. Data conversion module will be responsible for extracting data content from database which are selected, and then transforming them into a unified XML documents which will be presented to the user. Data load modules will be responsible for loading the data which are in XML documents to the target database. In the last section of paper, the feasibility of the interface is proved through an examination that realizing data migration from Access and SQL Server database to Oracle database.
引文
[1]樊顺利.钻井参数采集传输开环系统[J].石油钻探技术,1991,1(3):39-40
[2]姜昌勇.塔里木油田钻井数据无线通讯系统的现状与改进[J].钻采工艺,2000,23(4):88-89
[3]刘瑞文,郭学增.钻井数据实时远程传输系统研制[J].石油大学学报(自然科学版),2001,24(2):7-8
[4]王权,郭冬梅.将Intranet技术引入录井行业[J].录井技术,1997,12(1):12-13
[5]张进双,郭学增.海洋钻井完钻井数据集成服务系统[J].石油钻探技术,2002,30(1):7-9
[6]黎赵洪,李洪周.Advantage综合录井系统[J].录井技术,2002,13(4):13-17
[7]董新魁.DLS综合录井系统[J].录井技术,2001,12(3):22-25
[8]潘玮,张升品.Datalog综合录井仪[J].断块油气田,2001,11(5):51-53
[9]刘其春.综合录井仪[M].东营:胜利出版社,1986:13-56
[10]唐红军,万健.利用数据库转换实现异构数据库数据共享的研究与实现.计算机与数字工程,2005,6(33):99-102
[11]谢辉.浅谈录井信息的分类、意义、作用及应用[J].录井工程,2004,15(4):47-51
[12]荆心,雷聚超.异构数据库间数据迁移实现方法[J].现代电子技术,2004,20(9):1-2
[13]曹刚,周伯鑫.一种基于ODBC通用数据转换工具的实现[J].计算机工程与应用,1999,12(6):15-17
[14]张昊,倪笃明,陈定方.使用ODBC实现异构数据转换[J].武汉交通科技大学学报,1999,23(6):55-58
[15] Mark Rribeck. XML高级编程[M].北京:机械工业出版社,2002:116-118
[16]郭永峰,潘河杰编译.井场信息传输标准整合语言[J].录井技术,2004,15(1):31-33
[17]宁刚,张宁生,刘茜.基于WITSML的钻井数据文档设计实现[J].微机发展,2005,3(15):121-123
[18] Graff R,Seqrest R.Practical applications of a drilling data center [J].Oil and Gas Journal,1986,84(20):52-54
[19]李娜.基于XML的异构数据库访问中间件的研究与实现[D].郑州:郑州大学,2007:33-56
[20]黄伟婷.数据库中间件技术及应用[J].漳州师范学院学报,2006,18(2):25-30
[21]泽仁志玛,代光辉,陈会忠.数据库中间件技术探讨[J].地震地磁观测与研究,2005,26(1):67-72
[22]孙一林,彭波著.Java数据库编程实例[M].清华大学出版社,2003:56-97
[23]廖茂林,韦远霞.综合录井数据如何实现符合WITS要求的传输[J].录井技术,2001,4(12):59-62
[24]吴洁.XML应用教程[M].北京:清华大学出版社,2005:3-11
[25] XML Schema Daniela Florescu,Donald Kossmann.Storing and Querying XML Data using an RDMBS[J].IEEE Data Engineering Bulletin,1999,22(3):27-34
[26] Elliotte Rusty Harold著,刘文红、赵伟明等译.Java语言与XML处理教程[M].北京:电子工业出版社,2003:33-69
[27] AkmalB.Chaudhri等著.刑春晓等译.XML数据管理[M].北京:清华大学出版社,2006:56-77
[28] Daniela Florescu,Donald Kossmann.Storing and Querying XML Data using an RDMBS[J].IEEE Data Engineering Bulletin,1999,22(3):27-34.
[29]姜倩,武桂香,管凤旭.异构数据库间的动态数据迁移[J].应用科技,2005,32 (9):13-15
[30] Fujun Zhu , Turner , M.Dynamic data integration using Web services[C].IEEE International Conference.Los Alamitos:IEEE Computer Society,2004:262–269
[31]林清,董占球.网络环境下异构数据库交互访问中数据格式转换的研究[J].计算机应用与软件,2001,18(10):40-42
[32]高纯.XML数据环境下异构数据库间的数据交换方法[J].科技情报开发与经济,2004,14(8):241-242
[33]黄水源,罗胜,段隆振.利用XML技术实现异构数据库之间的数据转换[J].南昌大学学报(工科版),2005,11(3):85-88
[34] Peng,X.Brazile,R.Swigger,K.M.Extending XML Document Projection for Data Integration[C].Information Reuse and Integration , Conf , 2005.IRI-2005 IEEE International Conference. Piscataway:IEEE,2005:138-143
[35] Popfinger.C, Conrad.S.Maintaining global integrity in federated relational databasesusing interactive component systems[C].OTM Confederated International Conferences CoopIS,DOA,and ODBASE 2005. Berlin:pringer-Verlag,2005:539 -556
[36]潘传邦,杨瑞峰,王建军.Java实效编程百例[M].北京:人民邮电出版社,2003:108-141
[37] Abiteboul S,Clute S,Milo T,et al. Tools for Data Translation and Integration [C]. IEEE Data Engineering Bulletin,1999,22(1):3-8
[38]陈天煌,邹青梅.基于XML的异构数据库信息共享技术研究[J].武汉理工大学学报(交通科学与工程版),2005,14(2):129-132
[39] Natanya pitts.XML BLACK Book[M].北京:机械工业出版社,2001:124-136
[40] XML Schema Daniela Florescu,Donald Kossmann.Storing and Querying XML Data using an RDMBS[J].IEEE Data Engineering Bulletin,1999,22(3):27-34
[41] P Bohannon,J Freire.From XML Schema to Relations:A Cost-Based Approach to XML Storage [C] In:Proceedings of the 18th International Conference on Data Engineering,an Jose,California,USA,IEEE Computer Society,2002:64-92
[42]杨剑,唐慧佳,孙林夫,等.基于XML的异构数据库系统的研究与实现[J].计算机工程,2005,31(19):195-197
[43] Thomas G,Hartman B. Heterogeneous distributed database systems for production use [J].ACM Computing Surveys,1990,22(3):237-266
[44] Launders T,Prosenberg R. An overview of MULTIBASE [M]. Amsterdam, Netherlands:North Holland Publishing Company,1990:152-184
[45]姜利群.Java网络编程实例教程[M].北京:清华大学出版社,北京交通大学出版社,2004:175-181
[46] Microsoft Corporation.SQLServer2000体系结构与XML/Internet支持[M].北京:清华大学出版社,2001:34-65
[47]夏邦贵,刘凡馨.SQL Server数据库开发经典实例精解[M].北京:机械工作出版社,2006:40-68
[48]朱辉生.大型数据库系统概论(Oracle9i)[M].北京:高等教育出版社,2006:36-89
[49]荆棘鸟工作室.Oracle9i应用及开发实务[M].北京:中国水利水电出版社,2004:81-97
[50]徐慧慧,叶达峰.JBuilder编程技术与实例[M].人民邮电出版社,2006:28-74
[2]姜昌勇.塔里木油田钻井数据无线通讯系统的现状与改进[J].钻采工艺,2000,23(4):88-89
[3]刘瑞文,郭学增.钻井数据实时远程传输系统研制[J].石油大学学报(自然科学版),2001,24(2):7-8
[4]王权,郭冬梅.将Intranet技术引入录井行业[J].录井技术,1997,12(1):12-13
[5]张进双,郭学增.海洋钻井完钻井数据集成服务系统[J].石油钻探技术,2002,30(1):7-9
[6]黎赵洪,李洪周.Advantage综合录井系统[J].录井技术,2002,13(4):13-17
[7]董新魁.DLS综合录井系统[J].录井技术,2001,12(3):22-25
[8]潘玮,张升品.Datalog综合录井仪[J].断块油气田,2001,11(5):51-53
[9]刘其春.综合录井仪[M].东营:胜利出版社,1986:13-56
[10]唐红军,万健.利用数据库转换实现异构数据库数据共享的研究与实现.计算机与数字工程,2005,6(33):99-102
[11]谢辉.浅谈录井信息的分类、意义、作用及应用[J].录井工程,2004,15(4):47-51
[12]荆心,雷聚超.异构数据库间数据迁移实现方法[J].现代电子技术,2004,20(9):1-2
[13]曹刚,周伯鑫.一种基于ODBC通用数据转换工具的实现[J].计算机工程与应用,1999,12(6):15-17
[14]张昊,倪笃明,陈定方.使用ODBC实现异构数据转换[J].武汉交通科技大学学报,1999,23(6):55-58
[15] Mark Rribeck. XML高级编程[M].北京:机械工业出版社,2002:116-118
[16]郭永峰,潘河杰编译.井场信息传输标准整合语言[J].录井技术,2004,15(1):31-33
[17]宁刚,张宁生,刘茜.基于WITSML的钻井数据文档设计实现[J].微机发展,2005,3(15):121-123
[18] Graff R,Seqrest R.Practical applications of a drilling data center [J].Oil and Gas Journal,1986,84(20):52-54
[19]李娜.基于XML的异构数据库访问中间件的研究与实现[D].郑州:郑州大学,2007:33-56
[20]黄伟婷.数据库中间件技术及应用[J].漳州师范学院学报,2006,18(2):25-30
[21]泽仁志玛,代光辉,陈会忠.数据库中间件技术探讨[J].地震地磁观测与研究,2005,26(1):67-72
[22]孙一林,彭波著.Java数据库编程实例[M].清华大学出版社,2003:56-97
[23]廖茂林,韦远霞.综合录井数据如何实现符合WITS要求的传输[J].录井技术,2001,4(12):59-62
[24]吴洁.XML应用教程[M].北京:清华大学出版社,2005:3-11
[25] XML Schema Daniela Florescu,Donald Kossmann.Storing and Querying XML Data using an RDMBS[J].IEEE Data Engineering Bulletin,1999,22(3):27-34
[26] Elliotte Rusty Harold著,刘文红、赵伟明等译.Java语言与XML处理教程[M].北京:电子工业出版社,2003:33-69
[27] AkmalB.Chaudhri等著.刑春晓等译.XML数据管理[M].北京:清华大学出版社,2006:56-77
[28] Daniela Florescu,Donald Kossmann.Storing and Querying XML Data using an RDMBS[J].IEEE Data Engineering Bulletin,1999,22(3):27-34.
[29]姜倩,武桂香,管凤旭.异构数据库间的动态数据迁移[J].应用科技,2005,32 (9):13-15
[30] Fujun Zhu , Turner , M.Dynamic data integration using Web services[C].IEEE International Conference.Los Alamitos:IEEE Computer Society,2004:262–269
[31]林清,董占球.网络环境下异构数据库交互访问中数据格式转换的研究[J].计算机应用与软件,2001,18(10):40-42
[32]高纯.XML数据环境下异构数据库间的数据交换方法[J].科技情报开发与经济,2004,14(8):241-242
[33]黄水源,罗胜,段隆振.利用XML技术实现异构数据库之间的数据转换[J].南昌大学学报(工科版),2005,11(3):85-88
[34] Peng,X.Brazile,R.Swigger,K.M.Extending XML Document Projection for Data Integration[C].Information Reuse and Integration , Conf , 2005.IRI-2005 IEEE International Conference. Piscataway:IEEE,2005:138-143
[35] Popfinger.C, Conrad.S.Maintaining global integrity in federated relational databasesusing interactive component systems[C].OTM Confederated International Conferences CoopIS,DOA,and ODBASE 2005. Berlin:pringer-Verlag,2005:539 -556
[36]潘传邦,杨瑞峰,王建军.Java实效编程百例[M].北京:人民邮电出版社,2003:108-141
[37] Abiteboul S,Clute S,Milo T,et al. Tools for Data Translation and Integration [C]. IEEE Data Engineering Bulletin,1999,22(1):3-8
[38]陈天煌,邹青梅.基于XML的异构数据库信息共享技术研究[J].武汉理工大学学报(交通科学与工程版),2005,14(2):129-132
[39] Natanya pitts.XML BLACK Book[M].北京:机械工业出版社,2001:124-136
[40] XML Schema Daniela Florescu,Donald Kossmann.Storing and Querying XML Data using an RDMBS[J].IEEE Data Engineering Bulletin,1999,22(3):27-34
[41] P Bohannon,J Freire.From XML Schema to Relations:A Cost-Based Approach to XML Storage [C] In:Proceedings of the 18th International Conference on Data Engineering,an Jose,California,USA,IEEE Computer Society,2002:64-92
[42]杨剑,唐慧佳,孙林夫,等.基于XML的异构数据库系统的研究与实现[J].计算机工程,2005,31(19):195-197
[43] Thomas G,Hartman B. Heterogeneous distributed database systems for production use [J].ACM Computing Surveys,1990,22(3):237-266
[44] Launders T,Prosenberg R. An overview of MULTIBASE [M]. Amsterdam, Netherlands:North Holland Publishing Company,1990:152-184
[45]姜利群.Java网络编程实例教程[M].北京:清华大学出版社,北京交通大学出版社,2004:175-181
[46] Microsoft Corporation.SQLServer2000体系结构与XML/Internet支持[M].北京:清华大学出版社,2001:34-65
[47]夏邦贵,刘凡馨.SQL Server数据库开发经典实例精解[M].北京:机械工作出版社,2006:40-68
[48]朱辉生.大型数据库系统概论(Oracle9i)[M].北京:高等教育出版社,2006:36-89
[49]荆棘鸟工作室.Oracle9i应用及开发实务[M].北京:中国水利水电出版社,2004:81-97
[50]徐慧慧,叶达峰.JBuilder编程技术与实例[M].人民邮电出版社,2006:28-74