Scientific applications at exascale generate and analyze massive amounts of data.A critical requirement of these applications is the capability to access and manage this data efficiently on exascale *** I/O,the key te...
详细信息
Scientific applications at exascale generate and analyze massive amounts of data.A critical requirement of these applications is the capability to access and manage this data efficiently on exascale *** I/O,the key technology enables moving data between compute nodes and storage,faces monumental challenges from new applications,memory,and storage architectures considered in the designs of exascale *** the storage hierarchy is expanding to include node-local persistent memory,burst buffers,etc.,as well as disk-based storage,data movement among these layers must be *** I/O libraries of the future should be capable of handling file sizes of many terabytes and *** this paper,we describe new capabilities we have developed in Hierarchical Data Format version 5(HDF5),the most popular parallel I/O library for scientific ***5 is one of the most used libraries at the leadership computing facilities for performing parallel I/O on existing HPC *** state-of-the-art features we describe include:Virtual Object Layer(VOL),Data Elevator,asynchronous I/O,full-featured single-writer and multiple-reader(Full SWMR),and parallel *** this paper,we introduce these features,their implementations,and the performance and feature benefits to applications and other libraries.
Technology enhancements and the growing breadth of application workflows running on high-performance computing(HPC)platforms drive the development of new data services that provide high performance on these new platfo...
详细信息
Technology enhancements and the growing breadth of application workflows running on high-performance computing(HPC)platforms drive the development of new data services that provide high performance on these new platforms,provide capable and productive interfaces and abstractions for a variety of applications,and are readily adapted when new technologies are *** Mochi framework enables composition of specialized distributed data services from a collection of connectable modules and *** than forcing all applications to use a one-size-fits-all data staging and I/O software configuration,Mochi allows each application to use a data service specialized to its needs and access *** paper introduces the Mochi framework and *** Mochi core components and microservices are *** of the application of the Mochi methodology to the development of four specialized services are ***,a performance evaluation of a Mochi core component,a Mochi microservice,and a composed service providing an object model is *** paper concludes by positioning Mochi relative to related work in the HPC space and indicating directions for future work.
暂无评论