基于P2P技术的分布计算环境研究
摘要
计算科学，已经和传统的理论科学与试验科学并列成为第三门学科，彼此相辅相成地推动着人类科技发展和社会进步。然而随着人们求解问题领域的不断拓展，所遇到的问题也越来越复杂，而且规模也越来越大，解决这些问题所需要的计算能力也在大幅度提高。但是高性能并行计算机硬件昂贵，维护使用费用高昂，而现在的大多数计算机却还是价格低廉，配置性能较低的台式机，如何利用一种手段和机制，使得我们能够利用现有的基本硬件、网络环境和技术，来完成我们所需要的计算问题呢？可以利用基于分布式环境的并行计算来解决这个复杂的问题。利用网络中多台计算机互相协作，同时计算一个问题，这样的计算方式的出现就是一个必然，但是它的出现还是必须受到具体的技术条件的限制，需要建立一种机制和体系来解决这个问题。
P2P（Peer-to-Peer）计算系统是分布式的，有别于集中式的结构，也有别于基于服务器的结构。纯粹的P2P计算系统不存在不可或缺的服务器，在P2P计算系统中的实体一般同时扮演两种角色：客户和服务器。P2P技术可以为企业或者研究机构提供大规模的运算效能。它可以利用整个网络上的电脑，找寻闲置的中央处理器，内存以及磁盘空间，让企业或者研究机构将大型的运算工作分散到多部电脑上。此外，其运算结果可以直接分享到对等网络上的其他机器上。以往未曾运用的计算资源，如今可以不通过分布式建构就得到充分运用的效果。其结果就是更快速的计算过程以及更低廉的成本。
但是在一个纯粹的P2P计算的体系结构中，对等点如何彼此定位就是非常重要和必须的了。因为每个节点都是平等的，要完成有用的工作，P2P 应用程序中的对等点必须能够彼此发现对方和与对方交互。在本文中将简单的介绍在P2P 计算中，几种完成这一任务（称为发现）的方法，以及每种方法的优势和弱点。并且会针对一个点对点的开放源代码项目——JXTA[3]，进行P2P分布式计算的构架。
JXTA项目是由美国SUN Microsystems公司最先提出和开发的，现在已发展到了2.2版本，并且已经转交给一个非官方的组织JXTA.org进行维护和开发。它是一个面向Peer-to-Peer计算的开放式的网络计算平台，提供了一套开放的，基于Peer-to-Peer的协议，使得无论何种连接的设备，例如移动电话、PDA(个人数字助理)、PC机或者是服务器，都可以进行互相通讯和协作，并且JXTA分布计算系统的协议是与编程语言无关的，现在可以
采用的有Java语言版和C语言两种版本实现。
由于分布式网络的各组成节点在物理上或地理上的分散，造成了不可预测的信号传播的延迟、各个节点资源的负载能力不可预见性等，使得数值计算的算法实现要比集中式系统复杂，尤其是在分布计算资源的管理、待计算问题的调度和负载均衡等方面。而负载均衡策略及负载分配算法由于近年来集群服务和网络平台中间件飞速发展，成为了分布式算法研究方面的一个热点。本文提出了结合点对点网络的特点，利用启发式负载平衡法来实现的一个动态的分布式数值计算的负载分配方法。假定我们的生产者节点所发出的计算请求均为原子性请求，并且其解也为无交集的集合域，则我们可以利用如下的思路设计分布式数值计算的负载分配算法。在分布式网络中，一个组内节点（生产者）提交数值计算的请求时，先向组内的其它的节点（消费者）发出询问，通过多播的方式发送一个公告；如果是跨越多个网段而组成的节点组的话，或者有类似防火墙无法直接访问的情况，就可以使用一种分布式资源查找机制，结合利用HTTP隧道技术，穿越防火墙，实现动态的网络资源查找，进而来定位所发送的公告信息。公告内包含一个经过序列化的对象，此对象包括着一个简单计算方法，用来进行计算来测定收到公告节点的当前数值计算能力和负载能力。发送询问公告的同时，生产者分别记录组内消费者节点的发送时间戳，然后建立管道等待消费者节点的回执公告。消费者节点收到询问公告后，缓存公告到本地，然后序列化远端对象到当前节点内存空间，执行方法得到当前CPU的计算时间（秒的数目）。结束后产生包括运算时间和回执时间戳的回执公告，通过输出管道发送给生产者节点。生产者节点依此来评估各节点CPU的利用率和网络的相应时间，结合加权求值机制，对等待计算的任务进行动态分解，以期达到最大的负载均衡性。
本文首先介绍了当前分布式计算的状况，并深入探讨了P2P分布式计算的机制原理。在此基础上，根据对分布式计算网络的具体案例分析，提出以JXTA为底层架构，改进了基于管道的消息发布方法、基于处理器使用率的任务的分解机制，以及通过序列化的对象分发机制，实现了一个P2P的分布式计算的网络环境。这个机制体系包括三个机制：分布式索引机制、负载的动态分配机制和对象的分解组合机制。这三个机制的实现都是建立在P2P网络技术基础之上，进行改进而形成的。通过利用启发式负载平衡机制，充分结合每个节点的处理器的利用率情况，动态的分配每个节点的计算负载，尽量实现吞吐量最大、响应时间最短、负载又均匀的理想的负载平衡。通过对象序列化机制，实现了计算任务的分发处理，减少了网络中节点的广播通讯次数，避免了不必要的网络传输所带来的?
Recently years, the computing science is become more important than before. With the social development, the problem of computing we faced is much complex and large, and the spend that resolve it is expensive. One approach is that the high performance parallel computer as computing utility, but it’s use and maintenance is high cost and intricacy, these is the limit for the develop of the computing science. So it is necessary and important that find an easy and efficiency mechanism to resolve the current status.
P2P (Peer-to-Peer) computing system is distributed computing architecture, it is various of the B/S (Browser/Server) or C/S (Client/Server) model. As the Web continues to grow in both content and the number of connected devices, peer-to-peer computing is becoming increasingly popular. Popular software based on P2P technologies includes file sharing, distributed computing, and instant messenger services. Pure P2P system is not rely on one server or one special computer, so the one peer in the P2P network is role as both client side and server side, and the use of P2P technology, we can separate one huge computing task to the several computers for improve the spare PC’s utilize. At the same time, the result of computing can send to others peer in the network without any superfluity cost for the distributed computing environment. The result of use the P2P technology in the field of the huge scale computing is the high speed and low cost.
One coin has two sides you know, we enjoy the benefit of the P2P technology, and as the same of that has many hard steps should overcome. The things of how to find and locate each other is the need and import, for the every peer in the network the status is the same and equality. So if let the P2P exertion power and enjoy the benefit for us, should resolve the question that the peer fined each other and communicate with each other. This paper should discuss several methods about dispose these question, and the benefit and inferior position of these resolve scenario. Finally introduce one open source project faced to P2P technology, JXTA [3], and based on this project, building my own P2P distributed computing environment to settle the computing question.
JXTA is a set of open, generalized peer-to-peer (P2P) protocols that allow any connected device on the network —from cell phone to PDA, from PC to server
— to communicate and collaborate as peers. The JXTA protocols are independent of any programming language, and multiple implementations (called bindings in Project JXTA) exist for different environments. This document specifically discusses the Project JXTA binding on the Java? 2 Platform, Standard Edition software (J2SE?), and change the Project for fit me more.
Because of the distributed network is consist of separate peer, and every peer is geography or physics disperse, so induce the can not forecast the signal delay and dispose load ability, deduce the arithmetic of the distributed computing is much more complex than centralize system. Especially in the field of management of distributed computing resource, the distribution of the computing task and load balance. This paper advance one distributed computing system that combine the characteristic of the P2P network, based on enlighten to the load balance arithmetic. The detail step is that one peer send request advertisement for others peers used by the multicast or Shared Resource Distributed Index arithmetic, used the http tunnel technology through the firewall, if it is exist. The advertisement includes one serial object that own to one method. Correspond, the response peer return the compute value based on the CPU impose rate as the reason of separate the computing task. The advertisements the request peer requested should be cached in the local, the peers communicate with each other used by pipe based on the advertisements. We can subtype these advertisements to create our own subtypes toad an unlimited amount of additional and richer metadata information to each resource description, for the advertiseme
P2P（Peer-to-Peer）计算系统是分布式的，有别于集中式的结构，也有别于基于服务器的结构。纯粹的P2P计算系统不存在不可或缺的服务器，在P2P计算系统中的实体一般同时扮演两种角色：客户和服务器。P2P技术可以为企业或者研究机构提供大规模的运算效能。它可以利用整个网络上的电脑，找寻闲置的中央处理器，内存以及磁盘空间，让企业或者研究机构将大型的运算工作分散到多部电脑上。此外，其运算结果可以直接分享到对等网络上的其他机器上。以往未曾运用的计算资源，如今可以不通过分布式建构就得到充分运用的效果。其结果就是更快速的计算过程以及更低廉的成本。
但是在一个纯粹的P2P计算的体系结构中，对等点如何彼此定位就是非常重要和必须的了。因为每个节点都是平等的，要完成有用的工作，P2P 应用程序中的对等点必须能够彼此发现对方和与对方交互。在本文中将简单的介绍在P2P 计算中，几种完成这一任务（称为发现）的方法，以及每种方法的优势和弱点。并且会针对一个点对点的开放源代码项目——JXTA[3]，进行P2P分布式计算的构架。
JXTA项目是由美国SUN Microsystems公司最先提出和开发的，现在已发展到了2.2版本，并且已经转交给一个非官方的组织JXTA.org进行维护和开发。它是一个面向Peer-to-Peer计算的开放式的网络计算平台，提供了一套开放的，基于Peer-to-Peer的协议，使得无论何种连接的设备，例如移动电话、PDA(个人数字助理)、PC机或者是服务器，都可以进行互相通讯和协作，并且JXTA分布计算系统的协议是与编程语言无关的，现在可以
采用的有Java语言版和C语言两种版本实现。
由于分布式网络的各组成节点在物理上或地理上的分散，造成了不可预测的信号传播的延迟、各个节点资源的负载能力不可预见性等，使得数值计算的算法实现要比集中式系统复杂，尤其是在分布计算资源的管理、待计算问题的调度和负载均衡等方面。而负载均衡策略及负载分配算法由于近年来集群服务和网络平台中间件飞速发展，成为了分布式算法研究方面的一个热点。本文提出了结合点对点网络的特点，利用启发式负载平衡法来实现的一个动态的分布式数值计算的负载分配方法。假定我们的生产者节点所发出的计算请求均为原子性请求，并且其解也为无交集的集合域，则我们可以利用如下的思路设计分布式数值计算的负载分配算法。在分布式网络中，一个组内节点（生产者）提交数值计算的请求时，先向组内的其它的节点（消费者）发出询问，通过多播的方式发送一个公告；如果是跨越多个网段而组成的节点组的话，或者有类似防火墙无法直接访问的情况，就可以使用一种分布式资源查找机制，结合利用HTTP隧道技术，穿越防火墙，实现动态的网络资源查找，进而来定位所发送的公告信息。公告内包含一个经过序列化的对象，此对象包括着一个简单计算方法，用来进行计算来测定收到公告节点的当前数值计算能力和负载能力。发送询问公告的同时，生产者分别记录组内消费者节点的发送时间戳，然后建立管道等待消费者节点的回执公告。消费者节点收到询问公告后，缓存公告到本地，然后序列化远端对象到当前节点内存空间，执行方法得到当前CPU的计算时间（秒的数目）。结束后产生包括运算时间和回执时间戳的回执公告，通过输出管道发送给生产者节点。生产者节点依此来评估各节点CPU的利用率和网络的相应时间，结合加权求值机制，对等待计算的任务进行动态分解，以期达到最大的负载均衡性。
本文首先介绍了当前分布式计算的状况，并深入探讨了P2P分布式计算的机制原理。在此基础上，根据对分布式计算网络的具体案例分析，提出以JXTA为底层架构，改进了基于管道的消息发布方法、基于处理器使用率的任务的分解机制，以及通过序列化的对象分发机制，实现了一个P2P的分布式计算的网络环境。这个机制体系包括三个机制：分布式索引机制、负载的动态分配机制和对象的分解组合机制。这三个机制的实现都是建立在P2P网络技术基础之上，进行改进而形成的。通过利用启发式负载平衡机制，充分结合每个节点的处理器的利用率情况，动态的分配每个节点的计算负载，尽量实现吞吐量最大、响应时间最短、负载又均匀的理想的负载平衡。通过对象序列化机制，实现了计算任务的分发处理，减少了网络中节点的广播通讯次数，避免了不必要的网络传输所带来的?
Recently years, the computing science is become more important than before. With the social development, the problem of computing we faced is much complex and large, and the spend that resolve it is expensive. One approach is that the high performance parallel computer as computing utility, but it’s use and maintenance is high cost and intricacy, these is the limit for the develop of the computing science. So it is necessary and important that find an easy and efficiency mechanism to resolve the current status.
P2P (Peer-to-Peer) computing system is distributed computing architecture, it is various of the B/S (Browser/Server) or C/S (Client/Server) model. As the Web continues to grow in both content and the number of connected devices, peer-to-peer computing is becoming increasingly popular. Popular software based on P2P technologies includes file sharing, distributed computing, and instant messenger services. Pure P2P system is not rely on one server or one special computer, so the one peer in the P2P network is role as both client side and server side, and the use of P2P technology, we can separate one huge computing task to the several computers for improve the spare PC’s utilize. At the same time, the result of computing can send to others peer in the network without any superfluity cost for the distributed computing environment. The result of use the P2P technology in the field of the huge scale computing is the high speed and low cost.
One coin has two sides you know, we enjoy the benefit of the P2P technology, and as the same of that has many hard steps should overcome. The things of how to find and locate each other is the need and import, for the every peer in the network the status is the same and equality. So if let the P2P exertion power and enjoy the benefit for us, should resolve the question that the peer fined each other and communicate with each other. This paper should discuss several methods about dispose these question, and the benefit and inferior position of these resolve scenario. Finally introduce one open source project faced to P2P technology, JXTA [3], and based on this project, building my own P2P distributed computing environment to settle the computing question.
JXTA is a set of open, generalized peer-to-peer (P2P) protocols that allow any connected device on the network —from cell phone to PDA, from PC to server
— to communicate and collaborate as peers. The JXTA protocols are independent of any programming language, and multiple implementations (called bindings in Project JXTA) exist for different environments. This document specifically discusses the Project JXTA binding on the Java? 2 Platform, Standard Edition software (J2SE?), and change the Project for fit me more.
Because of the distributed network is consist of separate peer, and every peer is geography or physics disperse, so induce the can not forecast the signal delay and dispose load ability, deduce the arithmetic of the distributed computing is much more complex than centralize system. Especially in the field of management of distributed computing resource, the distribution of the computing task and load balance. This paper advance one distributed computing system that combine the characteristic of the P2P network, based on enlighten to the load balance arithmetic. The detail step is that one peer send request advertisement for others peers used by the multicast or Shared Resource Distributed Index arithmetic, used the http tunnel technology through the firewall, if it is exist. The advertisement includes one serial object that own to one method. Correspond, the response peer return the compute value based on the CPU impose rate as the reason of separate the computing task. The advertisements the request peer requested should be cached in the local, the peers communicate with each other used by pipe based on the advertisements. We can subtype these advertisements to create our own subtypes toad an unlimited amount of additional and richer metadata information to each resource description, for the advertiseme
引文
[1]陈国良，《并行计算-结构、算法、编程》，高等教育出版社 2000。
[2] Dreamtech,《Peer-to-Peer Application Development》,Publishing House of Electronics Industry 2002.
[3] The Project JXTA. http://www.jxta.org/.
[4] SETI@home, The Search for Extraterrestrial Intelligence (SETI), http://setiathome.ssl.berkeley.edu/
[5] The FightAIDS@Home. http://www.fightaidsathome.org/.
[6] Sun Microsystems,Inc. Project JXTA:An Open, Innovative Collaboration, http://www.jxta.org/. April 25, 2001
[7] Napster. http://www.napster.com/.
[8] Gnutella. http://www.gnutella.com/
[9] Bernard Traversat, Ahkil Arora. Project JXTA 2.0 Super-Peer Virtual Network. Sun Microsystems, Inc. 25 May 2003.
[10] Sun Microsystems,Inc. Project JXTA v2.0:Java? Programmer’s Guide. http://www.jxta.org , May 2003.
[11] Rod Johnson. Expert One-on-One: J2EE Design and Development. Wiley Publishing, Inc.? October 2002.
[12] Sun Microsystems,Inc. Project JXTA Technology:Creating Connected Communities. http://www.sun.com/. March 2003
[13] 鞠九滨, 《分布计算系统》， 高等教育出版社，1997.
[14] 程墨, 《网络优先集中式应用层多播及Java实现》， 吉林大学硕士论文，2003.
[15] 杨鲲， 《分布式系统中负载指标的确定及最佳机搜索算法的实现》， 硕士学位论文， 1995
[16] A.K.Ezzat ,”Load Balancing in NEST: A Network of Workstations”,Proc. Fall Joint Computer Conference, Dallas, TX . 1998.
[17] J.Ju,K.Yang and G.Xu,Using utilization as Load Index of Load Balancing,to be published.2000
[18] Logging Services Project@Apache .http://www.apache.org/. Apache Software Foundation .
[19] Jakarta Project@Apache . http://jakarta.apache.org/. Apache Software Foundation .
[2] Dreamtech,《Peer-to-Peer Application Development》,Publishing House of Electronics Industry 2002.
[3] The Project JXTA. http://www.jxta.org/.
[4] SETI@home, The Search for Extraterrestrial Intelligence (SETI), http://setiathome.ssl.berkeley.edu/
[5] The FightAIDS@Home. http://www.fightaidsathome.org/.
[6] Sun Microsystems,Inc. Project JXTA:An Open, Innovative Collaboration, http://www.jxta.org/. April 25, 2001
[7] Napster. http://www.napster.com/.
[8] Gnutella. http://www.gnutella.com/
[9] Bernard Traversat, Ahkil Arora. Project JXTA 2.0 Super-Peer Virtual Network. Sun Microsystems, Inc. 25 May 2003.
[10] Sun Microsystems,Inc. Project JXTA v2.0:Java? Programmer’s Guide. http://www.jxta.org , May 2003.
[11] Rod Johnson. Expert One-on-One: J2EE Design and Development. Wiley Publishing, Inc.? October 2002.
[12] Sun Microsystems,Inc. Project JXTA Technology:Creating Connected Communities. http://www.sun.com/. March 2003
[13] 鞠九滨, 《分布计算系统》， 高等教育出版社，1997.
[14] 程墨, 《网络优先集中式应用层多播及Java实现》， 吉林大学硕士论文，2003.
[15] 杨鲲， 《分布式系统中负载指标的确定及最佳机搜索算法的实现》， 硕士学位论文， 1995
[16] A.K.Ezzat ,”Load Balancing in NEST: A Network of Workstations”,Proc. Fall Joint Computer Conference, Dallas, TX . 1998.
[17] J.Ju,K.Yang and G.Xu,Using utilization as Load Index of Load Balancing,to be published.2000
[18] Logging Services Project@Apache .http://www.apache.org/. Apache Software Foundation .
[19] Jakarta Project@Apache . http://jakarta.apache.org/. Apache Software Foundation .
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |