Lessons Learned from Optimizing the Sunway Storage System for Higher Application I/O Performance

作     者：Qi Chen Kang Chen Zuo-Ning Chen Fellow CCF Wei Xue Xu Ji Bin Yang Qi Chen;Kang Chen;Zuo-Ning Chen;Wei Xue;Xu Ji;Bin Yang

作者机构：Department of Computer Science and TechnologyBeijing National Research Center for Information Science and Technology(BNRist)Tsinghua UniversityBeijing 100084China Chinese Academy of EngineeringBeijing 100088China National Supercomputing Center in WuxiWuxi 214100China School of SoftwareShandong UniversityJinan 250101China 

出 版 物：《Journal of Computer Science & Technology》 (计算机科学技术学报（英文版）)

年 卷 期：2020年第35卷第1期

页      面：47-60页

核心收录：

学科分类：08[工学] 0835[工学-软件工程] 081202[工学-计算机软件与理论] 0812[工学-计算机科学与技术（可授工学、理学学位）] 

基　　金：This work is supported by the National Key Research and Development Program of China under Grant No.2016YFB1000504 the Natural Science Foundation of China under Grant Nos.61433008,61373145,and 61572280,and China Postdoctoral Science Foundation under Grant No.2018M630162 

主　　题：high performance computing I/O interference parallel file system performance optimization resource misallocation 

摘      要：It is hard for applications to make full utilization of the peak bandwidth of the storage system in highperformance computers because of I/O interferences,storage resource misallocations and complex long I/O *** performed several studies to bridge this gap in the Sunway storage system,which serves the supercomputer Sunway *** locate these issues and connections between them,an end-to-end performance monitoring and diagnosis tool was developed to understand I/O behaviors of applications and the *** the help of the tool,we were about to find out the root causes of such performance barriers at the I/O forwarding layer and the parallel file system *** application-aware I/O forwarding allocation framework was used to address the I/O interferences and resource misallocations at the I/O forwarding layer.A performance-aware data placement mechanism was proposed to mitigate the impact of I/O interferences and performance variations of storage devices in the ***,applications obtained much better I/O *** the process,we also proposed a lightweight storage stack to shorten the I/O path of applications with N-N I/O *** paper summarizes these studies and presents the lessons learned from the process.