ExaHDF5: Delivering Efficient Parallel I/O on Exascale Computing Systems

作     者：Suren Byna M.Scot Breitenfeld Bin Dong Quincey Koziol Elena Pourmal Dana Robinson Jerome Soumagne Houjun Tang Venkatram Vishwanath Richard Warren Suren Byna;M.Scot Breitenfeld;Bin Dong;Quincey Koziol;Elena Pourmal;Dana Robinson;Jerome Soumagne;Houjun Tang;Venkatram Vishwanath;Richard Warren

作者机构：Lawrence Berkeley National LaboratoryBerkeleyCA 94597U.S.A The HDF GroupChampaignIL 61820U.S.A Argonne National LaboratoryLemontIL 60439U.S.A 

出 版 物：《Journal of Computer Science & Technology》 (计算机科学技术学报（英文版）)

年 卷 期：2020年第35卷第1期

页      面：145-160页

核心收录：

学科分类：0808[工学-电气工程] 08[工学] 0835[工学-软件工程] 0701[理学-数学] 0811[工学-控制科学与工程] 0812[工学-计算机科学与技术（可授工学、理学学位）] 081202[工学-计算机软件与理论] 

基　　金：This research was supported by the Exascale Computing Project under Grant No.17-SC-20-SC a joint project of the U.S.Department of Energy's Office of Science and National Nuclear Security Administration,responsible for delivering a capable exascale ecosystem,including software,applications,and hardware technology,to support the nation's exascale computing imperative This work is also supported by the Director,Office of Science,Office of Advanced Scientific Computing Research,of the U.S.Department of Energy under Contract Nos.DE-AC02-05CH11231 and DE-AC02-06CH11357 This research was funded in part by the Argonne Leadership Computing Facility which is a DOE Office of Science User Facility supported under Contract No.DE-AC02-06CH11357 This research used resources of the National Energy Research Scientific Computing Center which is DOE Office of Science User Facilities supported by the Office of Science of the U.S.Department of Energy under Contract No.DE-AC02-05CH11231 

主　　题：parallel I/O Hierarchical Data Format version 5(HDF5) I/O performance virtual object layer HDF5 optimizations 

摘      要：Scientific applications at exascale generate and analyze massive amounts of data.A critical requirement of these applications is the capability to access and manage this data efficiently on exascale *** I/O,the key technology enables moving data between compute nodes and storage,faces monumental challenges from new applications,memory,and storage architectures considered in the designs of exascale *** the storage hierarchy is expanding to include node-local persistent memory,burst buffers,etc.,as well as disk-based storage,data movement among these layers must be *** I/O libraries of the future should be capable of handling file sizes of many terabytes and *** this paper,we describe new capabilities we have developed in Hierarchical Data Format version 5(HDF5),the most popular parallel I/O library for scientific ***5 is one of the most used libraries at the leadership computing facilities for performing parallel I/O on existing HPC *** state-of-the-art features we describe include:Virtual Object Layer(VOL),Data Elevator,asynchronous I/O,full-featured single-writer and multiple-reader(Full SWMR),and parallel *** this paper,we introduce these features,their implementations,and the performance and feature benefits to applications and other libraries.