基于JXTA的P2P搜索网络研究
|
摘要
Peer-to-Peer(P2P)作为以文件共享为初始目的的应用，允许任意终端用户对等体(Peer)间通过Internet完成文件交换。在P2P应用仅有的短短几年发展时间里，它已成为了占用Internet流量的主要应用类型。P2P系统支持大量用户的能力已经开始显示出技术优势：它能够以较低的成本快速地部署强大的，大规模分布式应用。
一个P2P资源共享系统中，需要解决两个方面的问题：资源搜索和资源传送。由于P2P系统本身的分布式存储特点，使得实现具有扩展性的资源传送机制变得相对容易。根本的难点是如何发现拥有该资源的对等体，这也就是本文设法解决的问题。
当前的P2P系统，正在向参与系统的用户数目高速增加，对等体间性能和操作环境差异不断加大的趋势上发展。根据这种趋势，为充分发挥P2P体系的技术优点，本文采用混合Peer-to-Peer组网模式，在JXTA[2]平台上设计并实现了P2P搜索网络。
本文的主要贡献是，以扩展性，高效性，稳定性和负载平衡为目标，根据对等体性能的不同，在减少低性能对等体的搜索负载晕的同时，增加高性能对等体的搜索负载量，从而提升搜索网络的总体性能表现。其中，采用虚拟super-peer的自组，真实反映搜索网络的总体负载情况，并能在性能最高的对等体中产生合理数目的super-peer用于完成搜索服务，这适应了搜索网络扩展性的要求。采用客户资源索引在super-peer上的缓存机制和查询转发中的优化技术，以适应搜索网络的高效性要求。利用对等体记录自身运行环境的方法，以适应搜索网络的稳定性要求。利用super-peer间和super-peer与客户对等体间的查询重定向机制，以适应搜索网络的负载平衡要求。
Allowing file exchange among the end client peers through Internet is the initial purpose of Peer-to-Peer. In the short history of P2P application, it has become one of the main application types that consume a large fraction of Internet traffic. P2P architecture has begun to show its capability to support massive users, and this capability makes it suitable for rapidly deploy powerful and large-scale distributed applications with low cost
Resource searching and resource delivery are two crucial problems need to be solved in P2P system. The nature of distributed storage of P2P makes it easier to implement a scalable system for resource delivery. The key problem is how to locate the peer who has the resource, and it is also the problem this thesis tries to solve.
From the trend of P2P development, we can find the rapid increase of a huge new user population and the obvious differences between peers in capability and operating environment. To follow this trend and exploit the full potential of P2P architecture, the thesis chooses the hybrid Peer-to-Peer as networking mode, and based on JXTA platform, designes and implements a P2P search network.
The main contribution of the thesis is to satisfy the requirement about a great search method for a P2P system, that is scalability, efficiency, stability and balanced load. According to the difference of peer's capability, the algorithm decreases the search load of low capability peer and increases the search load of high capability peer to improve the aggregate performance of the search network.
To satisfy the requirement of scalability, the algorithm uses virtual super-peer self-organization to reliably reflect the search load of the network and choose a reasonable quantity of high capability peers to become super-peer. To satisfy the requirement of efficiency, the algorithm caches client resource index on super-peer and optimizes the forwarding of query message. To satisfy the requirement of stability, the algorithm records the run-time environment of peer. To satisfy the requirement of balanced load, the algorithm redirects the query message between super-peers and client peers.
一个P2P资源共享系统中，需要解决两个方面的问题：资源搜索和资源传送。由于P2P系统本身的分布式存储特点，使得实现具有扩展性的资源传送机制变得相对容易。根本的难点是如何发现拥有该资源的对等体，这也就是本文设法解决的问题。
当前的P2P系统，正在向参与系统的用户数目高速增加，对等体间性能和操作环境差异不断加大的趋势上发展。根据这种趋势，为充分发挥P2P体系的技术优点，本文采用混合Peer-to-Peer组网模式，在JXTA[2]平台上设计并实现了P2P搜索网络。
本文的主要贡献是，以扩展性，高效性，稳定性和负载平衡为目标，根据对等体性能的不同，在减少低性能对等体的搜索负载晕的同时，增加高性能对等体的搜索负载量，从而提升搜索网络的总体性能表现。其中，采用虚拟super-peer的自组，真实反映搜索网络的总体负载情况，并能在性能最高的对等体中产生合理数目的super-peer用于完成搜索服务，这适应了搜索网络扩展性的要求。采用客户资源索引在super-peer上的缓存机制和查询转发中的优化技术，以适应搜索网络的高效性要求。利用对等体记录自身运行环境的方法，以适应搜索网络的稳定性要求。利用super-peer间和super-peer与客户对等体间的查询重定向机制，以适应搜索网络的负载平衡要求。
Allowing file exchange among the end client peers through Internet is the initial purpose of Peer-to-Peer. In the short history of P2P application, it has become one of the main application types that consume a large fraction of Internet traffic. P2P architecture has begun to show its capability to support massive users, and this capability makes it suitable for rapidly deploy powerful and large-scale distributed applications with low cost
Resource searching and resource delivery are two crucial problems need to be solved in P2P system. The nature of distributed storage of P2P makes it easier to implement a scalable system for resource delivery. The key problem is how to locate the peer who has the resource, and it is also the problem this thesis tries to solve.
From the trend of P2P development, we can find the rapid increase of a huge new user population and the obvious differences between peers in capability and operating environment. To follow this trend and exploit the full potential of P2P architecture, the thesis chooses the hybrid Peer-to-Peer as networking mode, and based on JXTA platform, designes and implements a P2P search network.
The main contribution of the thesis is to satisfy the requirement about a great search method for a P2P system, that is scalability, efficiency, stability and balanced load. According to the difference of peer's capability, the algorithm decreases the search load of low capability peer and increases the search load of high capability peer to improve the aggregate performance of the search network.
To satisfy the requirement of scalability, the algorithm uses virtual super-peer self-organization to reliably reflect the search load of the network and choose a reasonable quantity of high capability peers to become super-peer. To satisfy the requirement of efficiency, the algorithm caches client resource index on super-peer and optimizes the forwarding of query message. To satisfy the requirement of stability, the algorithm records the run-time environment of peer. To satisfy the requirement of balanced load, the algorithm redirects the query message between super-peers and client peers.
引文
[1] B.Yang and H.Garcia-Molina. Improving efficiency of peer-to-peer search. In Proc. of the 28th Intl. Conf. on Dis-tributed Computing Systems, July 2002.
[2] Project JXTA website. http://www.jxta.org
[3] JXTA v1.0 Protocols Specification. Available at http://spec.jxta.org/v1.0/docbook/JXTAProtocols.html.
[4] Gnutella website. http://gnutella.wego.com
[5] IRC websitehttp://www.irc.org
[6] Freenet website. http://sourceforge.net/projects/freenet/
[7] Morpheus website. http://www.musiccity.com
[8] Open Napster website. http://www.opennapster.com
[9] Matei Ripeanu. Peer-to-Peer Architecture Case Study: Gnutella Network. Technical report University of Chicago. Available at http://www.cs.uchicago.edu/files/tr_authentic/TR-2001-26.pdf
[10] Bawa, Mayank, Garcia-Molina, and Hector. Dampened Broadcast in Peer-to-Peer Networks. Technical report, Stanford University, 2001. Available at http://dbpubs.stanford.edu/pub/2001-42
[11] Yatin Chawathe, Steven McCanne, and Eric Brewer. An architecture for internet content distribution as an infrastructure service, available at http://www.cs.berkeley.edu/yatin/papers, 2000
[12] B.Yang and H.Garcia-Molina. Comparing hybrid peer-to-peer systems. In Proc Of the 27th Intl. Conf. on Very Large Databases, September 2001.
[13] S.Ratnasamy, P.Francis, M.Handley, R.Karp, and S.Shenker. A scalable content-addressable network. In Proc. ACM SIGCOMM, August 2001
[14] Rudiger Schollmeier. A Definition of Peer-to-Peer Networking towards a Delimitation Against Classical Client-Server Concepts
[15] Beverly Yang and Hector Garcia-Molina. Designing a Super-Peer Network. Technical report, Stanford University, 2002. Available at http://www-db.stanford.edu/～byang/pubs/superpeer.pdf
[16] S.Saroiu, P.Gummadi, and S.Gribble. A measurement study of peer-to-peer file sharing systems. In Proc. of the Multimedia Computing and Networking, January 2002
[17] M.Faloutsos, P.Faloutsos, and C.Faloutsos. On Power-law Relationships of the Internet Topology. In Proceedings of SIGCOMM'99, Cambridge, MA, September 1999. ACM
[18] Miguel Castro, Peter Druschel, Y.Charlie Hu, and Antony Rowstron. Exploiting network proximity in peer-to-peer overlay networks. in FuDiCo 2002: International Workshop on Future Directions in Distributed Computing, June 2002
[19] Paul Francis, Sugih Jamin, Vern Paxson, Lixia Zhang, Daniel Gryniewicz, and Yixin Jin. An Architecture for a Global Internet Host Distance Estimation Service. In Proceedings IEEE Infocom'99, New York, NY, March 1999.
[20] Eugene Ng and Hui Zhang. Towards Global Network Positioning. In Proceedings of ACM SIGCOMM Internet Measurement Workshop, November 2001
[21] Sylvia Ratnasamy, Mark Handley, Richard Karp, and Scott Shenker. Topologically-aware overlay construction and server selection. In Proceedings of Infocom 2002, June 2002
[22] STOICA, I., MORRIS, R., KARGER, D., KAASHOEK, M.F., AND BALAKRISHNAN, H.Chord: A scalable peer-to-peer lookup service for internet applications. In Proc. SIGCOMM (2001)
[23] eDonkeys web site. http://www.edokey2000.com
[24] KaZaA web site. http://www.kazaa.com
[2] Project JXTA website. http://www.jxta.org
[3] JXTA v1.0 Protocols Specification. Available at http://spec.jxta.org/v1.0/docbook/JXTAProtocols.html.
[4] Gnutella website. http://gnutella.wego.com
[5] IRC websitehttp://www.irc.org
[6] Freenet website. http://sourceforge.net/projects/freenet/
[7] Morpheus website. http://www.musiccity.com
[8] Open Napster website. http://www.opennapster.com
[9] Matei Ripeanu. Peer-to-Peer Architecture Case Study: Gnutella Network. Technical report University of Chicago. Available at http://www.cs.uchicago.edu/files/tr_authentic/TR-2001-26.pdf
[10] Bawa, Mayank, Garcia-Molina, and Hector. Dampened Broadcast in Peer-to-Peer Networks. Technical report, Stanford University, 2001. Available at http://dbpubs.stanford.edu/pub/2001-42
[11] Yatin Chawathe, Steven McCanne, and Eric Brewer. An architecture for internet content distribution as an infrastructure service, available at http://www.cs.berkeley.edu/yatin/papers, 2000
[12] B.Yang and H.Garcia-Molina. Comparing hybrid peer-to-peer systems. In Proc Of the 27th Intl. Conf. on Very Large Databases, September 2001.
[13] S.Ratnasamy, P.Francis, M.Handley, R.Karp, and S.Shenker. A scalable content-addressable network. In Proc. ACM SIGCOMM, August 2001
[14] Rudiger Schollmeier. A Definition of Peer-to-Peer Networking towards a Delimitation Against Classical Client-Server Concepts
[15] Beverly Yang and Hector Garcia-Molina. Designing a Super-Peer Network. Technical report, Stanford University, 2002. Available at http://www-db.stanford.edu/～byang/pubs/superpeer.pdf
[16] S.Saroiu, P.Gummadi, and S.Gribble. A measurement study of peer-to-peer file sharing systems. In Proc. of the Multimedia Computing and Networking, January 2002
[17] M.Faloutsos, P.Faloutsos, and C.Faloutsos. On Power-law Relationships of the Internet Topology. In Proceedings of SIGCOMM'99, Cambridge, MA, September 1999. ACM
[18] Miguel Castro, Peter Druschel, Y.Charlie Hu, and Antony Rowstron. Exploiting network proximity in peer-to-peer overlay networks. in FuDiCo 2002: International Workshop on Future Directions in Distributed Computing, June 2002
[19] Paul Francis, Sugih Jamin, Vern Paxson, Lixia Zhang, Daniel Gryniewicz, and Yixin Jin. An Architecture for a Global Internet Host Distance Estimation Service. In Proceedings IEEE Infocom'99, New York, NY, March 1999.
[20] Eugene Ng and Hui Zhang. Towards Global Network Positioning. In Proceedings of ACM SIGCOMM Internet Measurement Workshop, November 2001
[21] Sylvia Ratnasamy, Mark Handley, Richard Karp, and Scott Shenker. Topologically-aware overlay construction and server selection. In Proceedings of Infocom 2002, June 2002
[22] STOICA, I., MORRIS, R., KARGER, D., KAASHOEK, M.F., AND BALAKRISHNAN, H.Chord: A scalable peer-to-peer lookup service for internet applications. In Proc. SIGCOMM (2001)
[23] eDonkeys web site. http://www.edokey2000.com
[24] KaZaA web site. http://www.kazaa.com