高度可扩展的3D叠前Kirchhoff时间偏移并行算法
|
摘要
为适应海量地震数据以及集群并行规模不断增大的趋势,提出了多维度成像空间分解算法.根据大规模集群系统有多个并行层次的特征,首先沿炮检距方向分解成像空间;然后再沿in-line方向继续切分,直到成像空间小于计算节点物理内存;最后在二维地表上以面元为单位分解成像空间.算法实现上,共炮检距成像空间映射到计算节点组上,计算节点内的CPU核之间按照round-robin均分面元.该并行算法在不增加数据通信量的情况下,降低了内存的需求,减少了通信开销和同步时间,提高了数据的局部性.实际资料测试表明,该并行算法比传统的输出并行和输入并行算法具备更好的性能与可扩展性,实验作业调度多达497个节点、7 552个线程,仍然具备较好的加速效果.
To support increasing survey sizes and processing complexity,we propose a practical approach that implements the large-scale parallel processing of 3D prestack Kirchhoff time migration(PKTM)on clusters of multi-core nodes.The parallel algorithm is based on three-level decomposition of the imaging space.Firstly,the imaging space is partitioned by offsets.Each node runs in just one process,and all processes are divided into several distinct groups.The imaging work of commonoffset space is assigned to a group,and the common-offset input traces are dynamically distributed to the processes of the group.Once all input traces are migrated,the local imaging sections of all the processes in a group are added to form the final common-offset image.In a node,the common-offset imaging section is further partitioned equally by common middle point(CMP)into as many blocks as the number of CPU cores,and the computing threads share the same input traces and spread the sampled points to a different set of imaging points.If the size of a common-offset imaging section exceeds the total physical memory on the compute node,the whole imaging space should be firstly partitioned along in-line direction so that each common-offset imaging space can fit in memory.The algorithm greatly reduces the memory requirement,does not introduce overlapping input traces between any processes, and makes it easy to implement fault-tolerance application. An implementation of the algorithm demonstrats high scalability and excellent performance in our experiment with actual data.Parallelism is scaled to efficiently use up to 497 nodes and 7552 threads.
To support increasing survey sizes and processing complexity,we propose a practical approach that implements the large-scale parallel processing of 3D prestack Kirchhoff time migration(PKTM)on clusters of multi-core nodes.The parallel algorithm is based on three-level decomposition of the imaging space.Firstly,the imaging space is partitioned by offsets.Each node runs in just one process,and all processes are divided into several distinct groups.The imaging work of commonoffset space is assigned to a group,and the common-offset input traces are dynamically distributed to the processes of the group.Once all input traces are migrated,the local imaging sections of all the processes in a group are added to form the final common-offset image.In a node,the common-offset imaging section is further partitioned equally by common middle point(CMP)into as many blocks as the number of CPU cores,and the computing threads share the same input traces and spread the sampled points to a different set of imaging points.If the size of a common-offset imaging section exceeds the total physical memory on the compute node,the whole imaging space should be firstly partitioned along in-line direction so that each common-offset imaging space can fit in memory.The algorithm greatly reduces the memory requirement,does not introduce overlapping input traces between any processes, and makes it easy to implement fault-tolerance application. An implementation of the algorithm demonstrats high scalability and excellent performance in our experiment with actual data.Parallelism is scaled to efficiently use up to 497 nodes and 7552 threads.
引文
[1]Bevc D.Imaging complex structures with semirecursive Kirchhoff migration[J].Geophysics,1997,62(2):577-588
[2]Huang Yi,Shi Xueming,Fan Jianke,et al.Review on parallel computing and its application in exploration geophysics[J].Progress in Geophys,2010,25(2):642-649(in Chinese)(黄易,师学明,范建柯,等.并行计算技术及其在勘探地球物理学中的现状与展望[J].地球物理学进展,2010,25(2):642-649)
[3]Panetta J,de Souza Filho R P R,da Cunha Filho C A,et al.Computational characteristics of production seismic migration and its performance on novel processor architectures[C]//Proc of the 19th Int Symp on Computer Architecture and High Performance Computing.Piscataway,NJ:IEEE,2007:11-18
[4]Liu Guofeng,Liu Hong,Wang Xiumin,et al.Two kinds of traveling time computation and parallel computing methods of kirchhoff migration[J].Progress in Geophys,2009,24(1):131-136(in Chinese)(刘国峰,刘洪,王秀闽,等.Kirchhoff积分时间偏移的两种走时计算及并行算法[J].地球物理学进展,2009,24(1):131-136)
[5]Shi X,Li C,Wang X,et al.A practical approach of curved ray prestack Kirchhoff time migration on GPGPU[G]//LNCS 5737:Proc of the 8th Int Symp on Advanced Parallel Processing Technologies.Berlin:Springer,2009:165-176
[6]Thakur R,Rabenseifner R,Gropp W.Optimization of collective communication operations in MPICH[J].International Journal of High Performance Computing Applications,2005,19(1):49-66
[7]Dai Hengchang.Parallel processing of prestack Kirchhoff time migration on a PC cluster[J].Computers&Geosciences,2005,31(7):891-899
[8]Li J,Hei D,Yan L.Partitioning algorithm of 3-D prestack parallel Kirchhoff depth migration for imaging spaces[C]//Proc of the 8th Int Conf on Grid and Cooperative Computing.Piscataway,NJ:IEEE,2009:276-280
[9]Panetta J,Teixeira T,de Souza Filho P R P,et al.Accelerating time and depth seismic migration by CPU and GPU cooperation[J].International Journal of Parallel Programming.2012,40(3):290-312
[10]Wang Huazhong,Cai Jiexiong,Kong Xiangning,et al.An implementation of Kirchhoff integral migration for large-scale data[J].Chinese Journal of Geophysics,2010,53(7):1699-1709(in Chinese)(王华忠,蔡杰雄,孔祥宁,等.适于大规模数据的三维Kirchhoff积分法体偏移实现方案[J].地球物理学报,2010,53(7):1699-1709)
[11]Zhao Gaishan.Current status and outlook of high performance computing for seismic exploration[J].Development and Application of High Performance Computing,2009,1(29):19-23(in Chinese)(赵改善.高性能计算在石油物探中的应用现状与前景[J].高性能计算发展与应用,2009,1(29):19-23)
[12]Wang Honglin.The revoluton of computer for geophysics[J].Progress in Exploration Geophysics,2009,32(4):233-238(in Chinese)(王宏琳.地球物理计算机的变革[J].勘探地球物理进展,2009,32(4):233-238)
[13]Chen Guoliang,Sun Guangzhong,Xu Yun,et al.Methodology of research on parallel algorithms[J].Chinese Journal of Computers,2008,31(9):1493-1502(in Chinese)(陈国良,孙广中,徐云,等.并行算法研究方法学[J].计算机学报,2008,31(9):1493-1502)
[14]Schroeder B,Gibson G A.A Large-scale study of failures in high-performance computing systems[J].IEEE Trans on Dependable and Secure Computing,2010,7(4):337-351
[15]Zhao Changhai,Yan Haihua,Shi Xiaohua,et al.DECF:A coarse-grained data-parallel programming framework for seismic processing[C]//Proc of the 1st Int Conf on Computer Science and Software Engineering.Piscataway,NJ:IEEE,2008:454-460
[16]Alexandrov A,Bratanov S,Fedorova J,et al.Parallelization made easier with Intel performance-tuning utility[J].Intel Technology Journal,2007,11(4):275-286
[17]Marr D T,Binns F,Hill D L,et al.Hyper-threading technology architecture and microarchitecture[J].Intel Technology Journal,2002,6(1):4-15
[18]Dean J,Ghemawat S.MapReduce:Simplified data processing on large clusters[J].Communications of the ACM,2008,51(1):107-113
[2]Huang Yi,Shi Xueming,Fan Jianke,et al.Review on parallel computing and its application in exploration geophysics[J].Progress in Geophys,2010,25(2):642-649(in Chinese)(黄易,师学明,范建柯,等.并行计算技术及其在勘探地球物理学中的现状与展望[J].地球物理学进展,2010,25(2):642-649)
[3]Panetta J,de Souza Filho R P R,da Cunha Filho C A,et al.Computational characteristics of production seismic migration and its performance on novel processor architectures[C]//Proc of the 19th Int Symp on Computer Architecture and High Performance Computing.Piscataway,NJ:IEEE,2007:11-18
[4]Liu Guofeng,Liu Hong,Wang Xiumin,et al.Two kinds of traveling time computation and parallel computing methods of kirchhoff migration[J].Progress in Geophys,2009,24(1):131-136(in Chinese)(刘国峰,刘洪,王秀闽,等.Kirchhoff积分时间偏移的两种走时计算及并行算法[J].地球物理学进展,2009,24(1):131-136)
[5]Shi X,Li C,Wang X,et al.A practical approach of curved ray prestack Kirchhoff time migration on GPGPU[G]//LNCS 5737:Proc of the 8th Int Symp on Advanced Parallel Processing Technologies.Berlin:Springer,2009:165-176
[6]Thakur R,Rabenseifner R,Gropp W.Optimization of collective communication operations in MPICH[J].International Journal of High Performance Computing Applications,2005,19(1):49-66
[7]Dai Hengchang.Parallel processing of prestack Kirchhoff time migration on a PC cluster[J].Computers&Geosciences,2005,31(7):891-899
[8]Li J,Hei D,Yan L.Partitioning algorithm of 3-D prestack parallel Kirchhoff depth migration for imaging spaces[C]//Proc of the 8th Int Conf on Grid and Cooperative Computing.Piscataway,NJ:IEEE,2009:276-280
[9]Panetta J,Teixeira T,de Souza Filho P R P,et al.Accelerating time and depth seismic migration by CPU and GPU cooperation[J].International Journal of Parallel Programming.2012,40(3):290-312
[10]Wang Huazhong,Cai Jiexiong,Kong Xiangning,et al.An implementation of Kirchhoff integral migration for large-scale data[J].Chinese Journal of Geophysics,2010,53(7):1699-1709(in Chinese)(王华忠,蔡杰雄,孔祥宁,等.适于大规模数据的三维Kirchhoff积分法体偏移实现方案[J].地球物理学报,2010,53(7):1699-1709)
[11]Zhao Gaishan.Current status and outlook of high performance computing for seismic exploration[J].Development and Application of High Performance Computing,2009,1(29):19-23(in Chinese)(赵改善.高性能计算在石油物探中的应用现状与前景[J].高性能计算发展与应用,2009,1(29):19-23)
[12]Wang Honglin.The revoluton of computer for geophysics[J].Progress in Exploration Geophysics,2009,32(4):233-238(in Chinese)(王宏琳.地球物理计算机的变革[J].勘探地球物理进展,2009,32(4):233-238)
[13]Chen Guoliang,Sun Guangzhong,Xu Yun,et al.Methodology of research on parallel algorithms[J].Chinese Journal of Computers,2008,31(9):1493-1502(in Chinese)(陈国良,孙广中,徐云,等.并行算法研究方法学[J].计算机学报,2008,31(9):1493-1502)
[14]Schroeder B,Gibson G A.A Large-scale study of failures in high-performance computing systems[J].IEEE Trans on Dependable and Secure Computing,2010,7(4):337-351
[15]Zhao Changhai,Yan Haihua,Shi Xiaohua,et al.DECF:A coarse-grained data-parallel programming framework for seismic processing[C]//Proc of the 1st Int Conf on Computer Science and Software Engineering.Piscataway,NJ:IEEE,2008:454-460
[16]Alexandrov A,Bratanov S,Fedorova J,et al.Parallelization made easier with Intel performance-tuning utility[J].Intel Technology Journal,2007,11(4):275-286
[17]Marr D T,Binns F,Hill D L,et al.Hyper-threading technology architecture and microarchitecture[J].Intel Technology Journal,2002,6(1):4-15
[18]Dean J,Ghemawat S.MapReduce:Simplified data processing on large clusters[J].Communications of the ACM,2008,51(1):107-113
版权所有：© 2023 中国地质图书馆 中国地质调查局地学文献中心