基于Web Service的分布式文件服务系统
|
摘要
基于Web Service构架的分布式文件服务系统基于Web、XML和SOAP技术、以及层次式缓存管理、分布式海量数据管理、访问控制等等多项技术，系统实现了企业级海量文件数据管理系统对可扩展、柔性、全分布、可靠、安全等几方面需求的支持。本文主要论述基于Web服务的分布式文件服务系统的关键技术及创新研究，包括体系结构、灵活的组件模型、全分布的本地文件存储单元、数据并发访问控制和海量数据高效的组织和检索等六个方面的研究成果。
论文的第一章首先介绍了分布式文件服务系统的起源、发展及将来的发展趋势，指出Web Service技术可作为实现企业分布式的海量文件存储和管理的的信息系统的开发平台和应用系统的集成平台。同时论文简单介绍了分布式文件服务系统的研究现状和几种典型产品的分析和比较，提出理想的分布式文件服务系统的具体要求。
论文的第二章介绍基于Web构架的分布式体系结构，基于对几种典型分布式文件服务系统的体系结构的优缺点的比较，论文提出系统的功能模型和体系结构，通过对系统功能的模块化，实现了系统的分布。为了提高系统的可扩展性，系统各个组件间通过XML格式的信息相互通讯，这一体系结构解决了现有的系统在扩展性差、对Web支持有限等方面的问题。
论文的第三章介绍基于Web Service的分布式文件服务系统的组件设计，构造出各个组件的模型。然后对组件元模型的许多概念给出了严格的定义，并以XML语言作为接口定义语言，这有利于系统的互操作性和与Web的集成。
论文的第四部分介绍了基于Web Service的分布式文件服务系统在应用实现中需要考虑的许多方面的问题，构造出针对四个方面问题的算法模型：集中控制单元设计模型、海量文件的本地存储策略、文件并发访问控制算法和分层的缓存管理机制。
第五部分，简要介绍了一个基于Web Service的分布式文件服务系统的原型系统的设计，并给出并分析了该应用系统的性能分析。
论文的最后总结论文的主要研究成果，并指出需进一步研究的工作。
Web Service based distributed file service system, which is based on Web, XML, and SOAP and built on hierarchical cache management, distributed mass-storage data management, access control, has a good extensibility, flexibility, reliability and security on the management system of enterprise's mass file date. This dissertation mainly introduces the key technology and innovative research of WS-DFS, such as web-based architecture, flexible component model, distributed local file storage unit, parallel access control and high efficiently organize and index for huge amount data.
In the 1st chapter, we firstly introduce the origin development and future of distribute file service system, then point out that WS-DFS is a kind of application platform for process and management huge amount file data、a kind of developing platform for flexible information system and a kind of application system integration framework. After introducing and comparing some typical DPS products, we put forward to a ideal DPS model and enumerate some detail rules of DPS design.
In the 2nd chapter, we discuss the architecture of our system. After analyzing the features of several famous DPS products, we bring forward WS-DFS system function model and architecture. After modularizing current system, we come to the distribution of system. In order to improve the system's expansibility, the modules in the system communicate through XML format information data, which can solve the bad expansibility and lack of support for Web Application in the current system.
In the 3rd chapter, we discuss the design of the component in the distribute file service system, and work out the models of components. Then we define some conceptions of the models and select XML as our interface definition language, which can benefit to the interaction and integration of system.
In the 4th chapter, we discuss the problems in the implement of the web service based distributed file service system. It focuses on four problems: the design of central control unit, the policy of mass storage in local file service unit, the parallel access control algorithm and hierarchical cache management mechanism.
In the 5th chapter, we describe the design of an instance of the web service based distribute file service system. Also we list the performance analysis of the system.
In the end,all of the works in this dissertation are summed up, and their futures are prospected.
论文的第一章首先介绍了分布式文件服务系统的起源、发展及将来的发展趋势，指出Web Service技术可作为实现企业分布式的海量文件存储和管理的的信息系统的开发平台和应用系统的集成平台。同时论文简单介绍了分布式文件服务系统的研究现状和几种典型产品的分析和比较，提出理想的分布式文件服务系统的具体要求。
论文的第二章介绍基于Web构架的分布式体系结构，基于对几种典型分布式文件服务系统的体系结构的优缺点的比较，论文提出系统的功能模型和体系结构，通过对系统功能的模块化，实现了系统的分布。为了提高系统的可扩展性，系统各个组件间通过XML格式的信息相互通讯，这一体系结构解决了现有的系统在扩展性差、对Web支持有限等方面的问题。
论文的第三章介绍基于Web Service的分布式文件服务系统的组件设计，构造出各个组件的模型。然后对组件元模型的许多概念给出了严格的定义，并以XML语言作为接口定义语言，这有利于系统的互操作性和与Web的集成。
论文的第四部分介绍了基于Web Service的分布式文件服务系统在应用实现中需要考虑的许多方面的问题，构造出针对四个方面问题的算法模型：集中控制单元设计模型、海量文件的本地存储策略、文件并发访问控制算法和分层的缓存管理机制。
第五部分，简要介绍了一个基于Web Service的分布式文件服务系统的原型系统的设计，并给出并分析了该应用系统的性能分析。
论文的最后总结论文的主要研究成果，并指出需进一步研究的工作。
Web Service based distributed file service system, which is based on Web, XML, and SOAP and built on hierarchical cache management, distributed mass-storage data management, access control, has a good extensibility, flexibility, reliability and security on the management system of enterprise's mass file date. This dissertation mainly introduces the key technology and innovative research of WS-DFS, such as web-based architecture, flexible component model, distributed local file storage unit, parallel access control and high efficiently organize and index for huge amount data.
In the 1st chapter, we firstly introduce the origin development and future of distribute file service system, then point out that WS-DFS is a kind of application platform for process and management huge amount file data、a kind of developing platform for flexible information system and a kind of application system integration framework. After introducing and comparing some typical DPS products, we put forward to a ideal DPS model and enumerate some detail rules of DPS design.
In the 2nd chapter, we discuss the architecture of our system. After analyzing the features of several famous DPS products, we bring forward WS-DFS system function model and architecture. After modularizing current system, we come to the distribution of system. In order to improve the system's expansibility, the modules in the system communicate through XML format information data, which can solve the bad expansibility and lack of support for Web Application in the current system.
In the 3rd chapter, we discuss the design of the component in the distribute file service system, and work out the models of components. Then we define some conceptions of the models and select XML as our interface definition language, which can benefit to the interaction and integration of system.
In the 4th chapter, we discuss the problems in the implement of the web service based distributed file service system. It focuses on four problems: the design of central control unit, the policy of mass storage in local file service unit, the parallel access control algorithm and hierarchical cache management mechanism.
In the 5th chapter, we describe the design of an instance of the web service based distribute file service system. Also we list the performance analysis of the system.
In the end,all of the works in this dissertation are summed up, and their futures are prospected.
引文
[1] Garth A.Gibson and Rodney Van Meter, "Network Attached Storage Architecture",Communications Of The ACM November 2000/Vol.43.No.11
[2] Makaroff D, Neufeld G, Hutchinson N "Design and implementation of a VBR continuous media file server", IEEE T SOFTWARE ENG 27(1): 13-28 JAN 2001
[3] James G. Mitchell Jeremy Dion, "A Comparison of Two Network-Based File Servers",1981 ACM 0-89791-062-1-12/81-00450
[4] HAN Hua, GUO Chaoyang, DAI Yafei, LI Xiaoming, "A Scheme to Construct Global File System", WISE2001 proceedings, IEEE Computer Society Press,Dec.2001
[5] 周志华，陈刚，董金祥 “基于Web的跨平台的分布多线程文件服务器设计”计算机工程 COMPUTER ENGINEER 2003 Vol. 29 No.4
[6] H.E. Sturgis, J.G.. Mitchell, and J.Israel, "Issues in the Design and Use of a Distributed File System", Op Sys Rev, 14(3), pp. 55-69, July 1980
[7] A.D.Birrell and R.M. Needham, "A Universal File Server", IEEE Trans Software Eng,SE-6(5) pp. 450-453, Sept. 1980
[8] Bruce Eckel, "Thinking in Java", Prentice-Hall Inc 2000
[9] Lee. E. and Thekkath, C. Petal, "Distributed Virtual Disks", ACM Press, New York, 1996,84-92
[10] Fabio Kon, "Distributed File Systems Past, Present and Future A Distributed File System for 2006"(1996)
[11] L. Svobodova, "File Servers for Network-Based Distributed Systems", ACM Computing Surveys, 16(4): 353--398, December 1984
[12] Sun Microsystems. NFS: Network file system version 3 protocol specification. Technical report, Sun Microsystems, Mountain View, CA, February 1994.
[13] Morris, J. H., Satyanarayanan, M., Conner, M.H., Howard, J.H., Rosenthal, D.S., Smith,F.D.Andrew, "A Distributed Personal Computing Environment" Communications of the ACM 29(3), March, 1986.
[14] Satyanarayanan, M, Kistler, J. J., Kumar, et. al., "Coda: a Highly available File System for a Distributed Workstation Environment," IEEE Trans. on Computers, 39(4): 447-459, 1990.
[15] J. Ousterhout, A. Cherenson, F. Douglis, M. Nelson, and B. Welch. "The Sprite network operating system" IEEE Computer, 21(2): 23--36, February 1988.
[16] J. Hartman and J. Ousterhout. "The Zebra Striped Network File System" Proc. 14-th Symposium on Operating Systems Principles, pages 29--43, December 1993.
[17] F. Oppacher and E. Suen. "HARP: A tableau-based theorem prover" Journal of Automated Reasoning, 4: 69--100, 1988
[18] Jed Krohnfeldt and Craig Steury: "Frolic: Logic Programming with Frobs", Utah PASS Project OpNote 96-08, 1993
[19] A. Birrell, A. Hisgen, C. Jerian, T. Mann, and G. Swart. The Echo Distributed File System.Technical Report 111, Digital Equipment Corp. Systems Research Center, September 1993
[20] M. Champion, C. Ferris, E. Newcomer, and D. Orchard. "Web Service Architecture", W3C Working Draft, 2002
[21] Seely, S. SOAP: Cross Platform "Web Service Development Using XML", Prentice Hall: Upper Saddle River, 2002
[22] J. Yang and M. Papazoglou, "Web Components: A Substrate for Web Service Reuse and Composition," in Proceedings of the 14th International Conference on Advanced Information Systems Engineering (CAiSE'02), Toronto, Canada, 2002
[23] Jeckle, M.: Web Service-Architecturen. Presentation slides at XML in Action 2002.Potsdam(2002).
[24] D. Florescu, A. Grunhagen, and D. Kossmann. XL: An XML language for Web service specification and composition. In Proceedings of the WWW'02 Conference, 2002.
[25] W. Hoschek. Peer-to-peer grid databases for web service discovery. To appear in Grid Computing: Making the Global Infrastructure a Reality, 2002
[26] WSDL (Web Service Definition Language), http://www.w3.org/2002/ws and www.w3.org/TR/wsdl
[27] Fensel, D. and C. Bussler, The Web Service Modeling Framework. Vrije Universiteit Amsterdam(VU) and Oracle Corporation
[28] J. Kubiatowicz, et al. "OceanStore: An Architecture for Global-Scale Persistent Storage"ASPLOS, December 2000.
[29] S. Rhea, P. Eaton, D. Geeis, H. Weatherspoon, B. Zhao, and J. Kubiatowicz. "Pond: The oceanstore prototype" In Proc. of USENIX File and Storage Technologies FAST, 2003.
[30] David Bindel, Yah Chert, Patrick Eaton, Dennis Geels, Ramakrishna Gummadi, Sean Rhea,Hakim Weatherspoon, Westley Weimer, Christopher Wells, Ben Zhao, and John Kubiatowicz. "Oceanstore: An extremely wide-area storage system", November 2000
[31] S. Rhea, C. Wells, P. Eaton, D. Geels, B. Zhao, H. Weatherspoon, and J. Kubiatowicz. "Maintenance fi'ee global storage in oceanstore". In Proc. of IEEE Internet Computing.IEEE, September 2001
[32] Curbera Francisco, Duftler Matthew, Khalaf Rania, Nagy William, Mukhi Nirmal,Weerawarana Sanjiva; "Unraveling the Web Services Web--An Introduction to SOAP,
WSDL, and UDDI" IEEE Internet Computing, March/April 2002
[33] 刘英丹，董传良，“利用Web Service实现企业应用集成” 《计算机应用》 第23卷 第7期，2003年7月
[34] F. DabeL M. F. Kaashoelq D. Karger, R. Morris, I. Stoica, "Wide-Area Cooperative Storage with CFS", SOSP, Oct 2001.
[35] M. Mizuno, J. Zhou, G. Singh. "An MWRW Based Strict Two Phase Lock Scheduler" In Proc. of the Int. Conf. On Computing and Information, iCCI'94. Peterborough, Canada,May 1994.
[36] 卢军，陈凌虎，卢显良．“一种适应WAN的新型分层缓存管理机制” 《电子科技大学学报》 第31卷第3期，2002年6月
[37] Lewontin, Steve and Mary Ellen Zurko. "The DCE Web Project: Providing Authorization and Other Distributed Services to the World Wide Web" OSF Research Institute.
[38] W. Rosenberry, D. Kenney, and G. Fisher. "Understanding DCE", O'Reilly and Associates,Inc., Sebastopal, California, 1993
[39] Open Software Foundation. "Introduction to OSF/DCE" Prentice-Hall, Englewood Cliffs,New Jersey, 1993
[40] Schill, A., Mock, M. "DC++: Distributed object-oriented system support on top of OSF DCE" Distributed Systems Engineering 1(2), 112-125, 1993
[41] M. Bever, K. Geihs, L. Heuser, M. Mulhauser, and A. Schill. "Distributed Systems, OSF DCE and Beyond" In A. Schill, editor, International DCE Workshop, number 731 in LNCS,Karlsruhe, October 7--8 1993. Springer Verlag
[2] Makaroff D, Neufeld G, Hutchinson N "Design and implementation of a VBR continuous media file server", IEEE T SOFTWARE ENG 27(1): 13-28 JAN 2001
[3] James G. Mitchell Jeremy Dion, "A Comparison of Two Network-Based File Servers",1981 ACM 0-89791-062-1-12/81-00450
[4] HAN Hua, GUO Chaoyang, DAI Yafei, LI Xiaoming, "A Scheme to Construct Global File System", WISE2001 proceedings, IEEE Computer Society Press,Dec.2001
[5] 周志华，陈刚，董金祥 “基于Web的跨平台的分布多线程文件服务器设计”计算机工程 COMPUTER ENGINEER 2003 Vol. 29 No.4
[6] H.E. Sturgis, J.G.. Mitchell, and J.Israel, "Issues in the Design and Use of a Distributed File System", Op Sys Rev, 14(3), pp. 55-69, July 1980
[7] A.D.Birrell and R.M. Needham, "A Universal File Server", IEEE Trans Software Eng,SE-6(5) pp. 450-453, Sept. 1980
[8] Bruce Eckel, "Thinking in Java", Prentice-Hall Inc 2000
[9] Lee. E. and Thekkath, C. Petal, "Distributed Virtual Disks", ACM Press, New York, 1996,84-92
[10] Fabio Kon, "Distributed File Systems Past, Present and Future A Distributed File System for 2006"(1996)
[11] L. Svobodova, "File Servers for Network-Based Distributed Systems", ACM Computing Surveys, 16(4): 353--398, December 1984
[12] Sun Microsystems. NFS: Network file system version 3 protocol specification. Technical report, Sun Microsystems, Mountain View, CA, February 1994.
[13] Morris, J. H., Satyanarayanan, M., Conner, M.H., Howard, J.H., Rosenthal, D.S., Smith,F.D.Andrew, "A Distributed Personal Computing Environment" Communications of the ACM 29(3), March, 1986.
[14] Satyanarayanan, M, Kistler, J. J., Kumar, et. al., "Coda: a Highly available File System for a Distributed Workstation Environment," IEEE Trans. on Computers, 39(4): 447-459, 1990.
[15] J. Ousterhout, A. Cherenson, F. Douglis, M. Nelson, and B. Welch. "The Sprite network operating system" IEEE Computer, 21(2): 23--36, February 1988.
[16] J. Hartman and J. Ousterhout. "The Zebra Striped Network File System" Proc. 14-th Symposium on Operating Systems Principles, pages 29--43, December 1993.
[17] F. Oppacher and E. Suen. "HARP: A tableau-based theorem prover" Journal of Automated Reasoning, 4: 69--100, 1988
[18] Jed Krohnfeldt and Craig Steury: "Frolic: Logic Programming with Frobs", Utah PASS Project OpNote 96-08, 1993
[19] A. Birrell, A. Hisgen, C. Jerian, T. Mann, and G. Swart. The Echo Distributed File System.Technical Report 111, Digital Equipment Corp. Systems Research Center, September 1993
[20] M. Champion, C. Ferris, E. Newcomer, and D. Orchard. "Web Service Architecture", W3C Working Draft, 2002
[21] Seely, S. SOAP: Cross Platform "Web Service Development Using XML", Prentice Hall: Upper Saddle River, 2002
[22] J. Yang and M. Papazoglou, "Web Components: A Substrate for Web Service Reuse and Composition," in Proceedings of the 14th International Conference on Advanced Information Systems Engineering (CAiSE'02), Toronto, Canada, 2002
[23] Jeckle, M.: Web Service-Architecturen. Presentation slides at XML in Action 2002.Potsdam(2002).
[24] D. Florescu, A. Grunhagen, and D. Kossmann. XL: An XML language for Web service specification and composition. In Proceedings of the WWW'02 Conference, 2002.
[25] W. Hoschek. Peer-to-peer grid databases for web service discovery. To appear in Grid Computing: Making the Global Infrastructure a Reality, 2002
[26] WSDL (Web Service Definition Language), http://www.w3.org/2002/ws and www.w3.org/TR/wsdl
[27] Fensel, D. and C. Bussler, The Web Service Modeling Framework. Vrije Universiteit Amsterdam(VU) and Oracle Corporation
[28] J. Kubiatowicz, et al. "OceanStore: An Architecture for Global-Scale Persistent Storage"ASPLOS, December 2000.
[29] S. Rhea, P. Eaton, D. Geeis, H. Weatherspoon, B. Zhao, and J. Kubiatowicz. "Pond: The oceanstore prototype" In Proc. of USENIX File and Storage Technologies FAST, 2003.
[30] David Bindel, Yah Chert, Patrick Eaton, Dennis Geels, Ramakrishna Gummadi, Sean Rhea,Hakim Weatherspoon, Westley Weimer, Christopher Wells, Ben Zhao, and John Kubiatowicz. "Oceanstore: An extremely wide-area storage system", November 2000
[31] S. Rhea, C. Wells, P. Eaton, D. Geels, B. Zhao, H. Weatherspoon, and J. Kubiatowicz. "Maintenance fi'ee global storage in oceanstore". In Proc. of IEEE Internet Computing.IEEE, September 2001
[32] Curbera Francisco, Duftler Matthew, Khalaf Rania, Nagy William, Mukhi Nirmal,Weerawarana Sanjiva; "Unraveling the Web Services Web--An Introduction to SOAP,
WSDL, and UDDI" IEEE Internet Computing, March/April 2002
[33] 刘英丹，董传良，“利用Web Service实现企业应用集成” 《计算机应用》 第23卷 第7期，2003年7月
[34] F. DabeL M. F. Kaashoelq D. Karger, R. Morris, I. Stoica, "Wide-Area Cooperative Storage with CFS", SOSP, Oct 2001.
[35] M. Mizuno, J. Zhou, G. Singh. "An MWRW Based Strict Two Phase Lock Scheduler" In Proc. of the Int. Conf. On Computing and Information, iCCI'94. Peterborough, Canada,May 1994.
[36] 卢军，陈凌虎，卢显良．“一种适应WAN的新型分层缓存管理机制” 《电子科技大学学报》 第31卷第3期，2002年6月
[37] Lewontin, Steve and Mary Ellen Zurko. "The DCE Web Project: Providing Authorization and Other Distributed Services to the World Wide Web" OSF Research Institute.
[38] W. Rosenberry, D. Kenney, and G. Fisher. "Understanding DCE", O'Reilly and Associates,Inc., Sebastopal, California, 1993
[39] Open Software Foundation. "Introduction to OSF/DCE" Prentice-Hall, Englewood Cliffs,New Jersey, 1993
[40] Schill, A., Mock, M. "DC++: Distributed object-oriented system support on top of OSF DCE" Distributed Systems Engineering 1(2), 112-125, 1993
[41] M. Bever, K. Geihs, L. Heuser, M. Mulhauser, and A. Schill. "Distributed Systems, OSF DCE and Beyond" In A. Schill, editor, International DCE Workshop, number 731 in LNCS,Karlsruhe, October 7--8 1993. Springer Verlag