检索结果-南通市图书馆

I/O Acceleration via Multi-Tiered Data Buffering and Prefetching

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Journal of Computer Science & Technology 2020年第1期35卷 92-120页

作者： Anthony Kougkas Hariharan Devarajan Xian-He Sun Department of Computer Science Illinois Institute of TechnologyChicago 60616U.S.A

Modern High-Performance Computing(HPC)systems are adding extra layers to the memory and storage hierarchy,named deep memory and storage hierarchy(DMSH),to increase I/O performance.New hardware technologies,such as NVMe and SSD,have been introduced in burst buffer installations to reduce the pressure for external storage and boost the burstiness of modern I/O systems.The DMSH has demonstrated its strength and potential in practice.However,each layer of DMSH is an independent heterogeneous system and data movement among more layers is significantly more complex even without considering heterogeneity.How to efficiently utilize the DMSH is a subject of research facing the HPC community.Further,accessing data with a high-throughput and low-latency is more imperative than ever.Data prefetching is a well-known technique for hiding read latency by requesting data before it is needed to move it from a high-latency medium(e.g.,disk)to a low-latency one(e.g.,main memory).However,existing solutions do not consider the new deep memory and storage hierarchy and also suffer from under-utilization of prefetching resources and unnecessary evictions.Additionally,existing approaches implement a client-pull model where understanding the application's I/O behavior drives prefetching decisions.Moving towards exascale,where machines run multiple applications concurrently by accessing files in a workflow,a more data-centric approach resolves challenges such as cache pollution and redundancy.in this paper,we present the design and implementation of Hermes:a new,heterogeneous-aware,multi-tiered,dynamic,and distributed I/O buffering system.Hermes enables,manages,supervises,and,in some sense,extends I/O buffering to fully integrate into the DMSH.We introduce three novel data placement policies to efficiently utilize all layers and we present three novel techniques to perform memory,metadata,and communication management in hierarchical buffering systems.Additionally,we demonstrate the benefits of a truly hierarchical data prefetcher that ado

关键词： I/O buffering heterogeneous buffering layered buffering deep memory hierarchy burst buffers hierarchical data prefetching data-centric architecture

ExaHDF5: Delivering Efficient Parallel I/O on Exascale Computing Systems

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Journal of Computer Science & Technology 2020年第1期35卷 145-160页

作者： Suren Byna M.Scot Breitenfeld Bin Dong Quincey Koziol Elena Pourmal Dana Robinson Jerome Soumagne Houjun Tang Venkatram Vishwanath Richard Warren Lawrence Berkeley National Laboratory BerkeleyCA 94597U.S.A The HDF Group ChampaignIL 61820U.S.A Argonne National Laboratory LemontIL 60439U.S.A

Scientific applications at exascale generate and analyze massive amounts of data.A critical requirement of these applications is the capability to access and manage this data efficiently on exascale systems.Parallel I/O,the key technology enables moving data between compute nodes and storage,faces monumental challenges from new applications,memory,and storage architectures considered in the designs of exascale systems.As the storage hierarchy is expanding to include node-local persistent memory,burst buffers,etc.,as well as disk-based storage,data movement among these layers must be efficient.Parallel I/O libraries of the future should be capable of handling file sizes of many terabytes and beyond.in this paper,we describe new capabilities we have developed in Hierarchical Data Format version 5(HDF5),the most popular parallel I/O library for scientific applications.HDF5 is one of the most used libraries at the leadership computing facilities for performing parallel I/O on existing HPC systems.The state-of-the-art features we describe include:Virtual object Layer(VoL),Data Elevator,asynchronous I/O,full-featured single-writer and multiple-reader(Full SWMR),and parallel querying.in this paper,we introduce these features,their implementations,and the performance and feature benefits to applications and other libraries.

关键词： parallel I/O Hierarchical Data Format version 5(HDF5) I/O performance virtual object layer HDF5 optimizations

PAM: an efficient power-aware multilevel cache policy to reduce energy consumption of storage systems

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Frontiers of Computer Science 2019年第4期13卷 850-863页

作者： Xiaodong MENG Chentao WU Minyi GUo Long ZHENG Jingyu ZHANG Shanghai Key Laboratory of Scalable Computing and Systems Department of Computer Science & Engineering Shanghai Jiao Tong University Shanghai 200240 China

Energy consumption is one of the most significant aspects of large-scale storage systems where multilevel caches are widely used. in a typical hierarchical storage structure, upper-level storage serves as a cache for the lower level, forming a distributed multilevel cache system. in the past two decades, several classic LRU-based multilevel cache policies have been proposed to improve the overall I/O performance of storage systems. However, few power-aware multilevel cache policies focus on the storage devices in the bottom level, which consume more than 27% of the energy of the whole system [1]. To address this problem, we propose a novel power-aware multilevel cache (PAM) policy that can reduce the energy consumption of high-performance and I/O bandwidth storage devices. in our PAM policy, an appropriate number of cold dirty blocks in the upper level cache are identified and selected to flush directly to the storage devices, providing high probability extension of the lifetime of disks in standby mode. To demonstrate the effectiveness of our proposed policy, we conduct several simulations with real-world traces. Compared to existing popular cache schemes such as PA? LRU, PB-LRU, and Demote, PAM reduces power consumption by up to 15% under different I/O workloads, and improves energy efficiency by up to 50.5%.

关键词： storage system multilevel cache energy consumption I/O performance hint

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

流式处理的异步图处理框架

软件学报 2018年第3期29卷 528-544页

作者：李金吉张岩峰巩树凤于戈高立新东北大学计算机科学与工程学院辽宁沈阳110819 Department of Electrical and Computer Engineering University of Massachusetts Amherst

分布式图计算是目前处理大图数据的主流技术,但是存在诸多无法避免的问题,比如分布式计算的负载均衡和分布式实现的调试和优化仍然非常困难.另一方面,近几年的研究结果表明:通过设计合理的数据结构和处理模型,在单个PC上基于大容量磁盘... 详细信息

分布式图计算是目前处理大图数据的主流技术,但是存在诸多无法避免的问题,比如分布式计算的负载均衡和分布式实现的调试和优化仍然非常困难.另一方面,近几年的研究结果表明:通过设计合理的数据结构和处理模型,在单个PC上基于大容量磁盘的大图计算往往可以获得与分布式图计算相当的处理性能.例如,Graph Chi在单机上的处理性能与Spark在50台节点上的处理性能相差无几.结合累加迭代计算和单机并行处理技术,提出流式处理的异步计算模型ASP.它实现了对磁盘的完全顺序访问,允许流式的顺序载入结构数据的同时进行异步更新计算.基于ASP模型,提出了一种流式处理的异步图处理框架S-Maiter,实现了高效率的基于外存的单机大图处理,通过I/O线程优化、内存资源监控、shard级优先级调度等优化技术,提高了系统处理大图数据的性能.实验结果表明:在处理大图数据(1 300万顶点,5亿连边)时,仅仅需要1台PC机计算资源的S-Maiter与在16台PC上运行的分布式Maiter的性能几乎相当.并且,S-Maiter比另外一个流行的单机大图处理系统Graph Chi要快1.5倍.

关键词：外存异步累加模型 I/O 流式处理

Mochi: Composing Data Services for High-Performance Computing Environments

在线全文

学校读者我要写书评

暂无评论

Journal of Computer Science & Technology 2020年第1期35卷 121-144页

作者： Robert BRoss George Amvrosiadis Philip Carns Charles DCranor Matthieu Dorier Kevin Harms Greg Ganger Garth Gibson Samuel KGutierrez Robert Latham Bob Robey Dana Robinson Bradley Settlemyer Galen Shipman Shane Snyder Jerome Soumagne Qing Zheng Argonne National Laboratory LemontIL 60439U.S.A Parallel Data Laboratory Carnegie Mellon UniversityPittsburghPA 15213U.S.A Vector institute for Artificial intelligence TorontoOntarioCanada Los Alamos National Laboratory Los Alamos NMU.S.A The HDF Group Champaign ILU.S.A

Technology enhancements and the growing breadth of application workflows running on high-performance computing(HPC)platforms drive the development of new data services that provide high performance on these new platforms,provide capable and productive interfaces and abstractions for a variety of applications,and are readily adapted when new technologies are deployed.The Mochi framework enables composition of specialized distributed data services from a collection of connectable modules and subservices.Rather than forcing all applications to use a one-size-fits-all data staging and I/O software configuration,Mochi allows each application to use a data service specialized to its needs and access patterns.This paper introduces the Mochi framework and methodology.The Mochi core components and microservices are described.Examples of the application of the Mochi methodology to the development of four specialized services are detailed.Finally,a performance evaluation of a Mochi core component,a Mochi microservice,and a composed service providing an object model is performed.The paper concludes by positioning Mochi relative to related work in the HPC space and indicating directions for future work.

关键词： storage and I/O data-intensive computing distributed services high-performance computing

Regulation of isobutane/1-butene adsorption behaviors on the acidic ionic liquids-functionalized MCM-22 zeolite

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Chinese Journal of Chemical Engineering 2018年第1期26卷 127-136页

作者： Keting Jin Tao Zhang Shaojun Yuan Shengwei Tang Multi-phase Mass Transfer and Reaction Engineering Lab College of Chemical EngineeringSichuan UniversityChengdu 610065China

The adsorption ratio of isobutane/1-butene on the catalyst surface is one of the most important factors for the C_4 alkylation process. Regulation of isobutane/1-butene adsorption ratio on the zeolite-supported acid catalyst is a big challenge for catalyst preparation. To regulate the isobutane/1-butene adsorption ratio, four types of ionic liquid(i.e., iL) with different alkyl chain lengths and different acid group numbers were synthesized and were subsequently immobilized onto the MCM-22 zeolite. The as-synthesized iL-immobilized MCM-22(i.e., MCM-22-iL)was characterized by FTiR, TGA, BET, XPS and XRD, and their adsorption capacities and adsorption molar ratios of isobutane to 1-butene(I/O) were investigated to correlate with surface features of MCM-22-iL. Results showed that the immobilization of iLs led to a decrease of specific surface area and pore volume. But the surface density of acid groups was increased and the adsorption molar ratio of isobutane/1-butene(I/O) was significantly improved by the immobilization of ionic liquids. The adsorption molar ratio of I/O is substantially improved from 0.75 to above 0.9 at 300 kPa upon immobilizing iLs. Although the alkyl chain length of iLs was found to have little effect on the adsorption molar ratio of I/O, the increase of acid group numbers led to a dramatic decrease in the adsorption I/O ratio. The results illustrated that immobilizing ionic liquids is an effective way to modify the textural, chemical and morphological properties of MCM-22. Accordingly, the immobilization of ionic liquids provides a novel and a feasible way to regulate the adsorption I/O ratio on an adsorbent or a solid catalyst.

关键词： MCM-22 吸附行为离子沸石催化剂表面 I/O 酸催化剂吸附能力

MUSE:一种面向云存储系统的高性能元数据存储引擎

在线全文

学校读者我要写书评

暂无评论

电子科技大学学报 2016年第2期45卷 221-226页

作者：段翰聪向小可吕鹏程电子科技大学计算机科学与工程学院成都611731

该文设计了一种高性能的面向云存储系统的元数据存储引擎(MUSE)。首先,其底层物理存储模块采用LSM-tree模型的高速key-value存储引擎Level DB方案,通过设计多缓存表和多线程紧凑机制对该方案进行优化,使其可以充分利用内存和多核CPU并... 详细信息

该文设计了一种高性能的面向云存储系统的元数据存储引擎(MUSE)。首先,其底层物理存储模块采用LSM-tree模型的高速key-value存储引擎Level DB方案,通过设计多缓存表和多线程紧凑机制对该方案进行优化,使其可以充分利用内存和多核CPU并行能力;其次,提出了基于多I/O通道的元数据存取调度机制。通道之间读写操作隔离,聚合多个通道为上层提供高并发随机I/O读写能力;此外,针对上层目录命名空间管理,提出路径分割映射和全路径映射策略两种策略,可基于不同的应用场景在性能与可用性间进行折中选择。系统测试结果表明,MUSE能够很好地适应海量小文件存储场景,相对于其他元数据存储系统在性能上有显著的提升。

关键词： I/O LSM-tree 海量元数据性能小文件存储引擎

在线全文

学校读者我要写书评

暂无评论

中国环境科学 2015年第4期35卷 1032-1039页

作者：刘江海白志鹏韩斌刘雅婷游燕张杰峰周剑中国环境科学研究院环境基准与风险评估国家重点实验室北京100012 南开大学环境科学与工程学院天津300071

为研究室内外PM10中多环芳烃相关性和来源特征,于2009年非采暖季、采暖季在天津市某小区36位住户室内和室外进行PM10膜采样,并分析其中12种PAHs浓度.结果表明,非采暖季较采暖季更为良好的室内外通风性,导致非采暖季室内外PM10中PAHs浓... 详细信息

为研究室内外PM10中多环芳烃相关性和来源特征,于2009年非采暖季、采暖季在天津市某小区36位住户室内和室外进行PM10膜采样,并分析其中12种PAHs浓度.结果表明,非采暖季较采暖季更为良好的室内外通风性,导致非采暖季室内外PM10中PAHs浓度没有显著差异（P〉0.05）,而采暖季室内PM10中PAHs浓度显著小于室外浓度;非采暖季室内外PM10中各PAHs占总PAHs的质量百分比基本一致,而采暖季其室内外质量百分比有明显不同;采暖季和非采暖季室内外PM10中PAHs的I/O平均值均低于1,但采暖季I/O值小于非采暖季.对参与者的时间活动模式分析表明,在采暖季和非采暖季,室内吸烟、清洁活动以及烹饪对室内PM10中PAHs浓度变化均没有显著影响（P〉0.05）.室内外PAHs浓度线性回归分析表明,室内PM10中5～7环PAHs主要受到室外源的影响,2～4环PAHs主要受到室内源的影响,非采暖季和采暖季室内外PM10中总PAHs的有效穿透因子分别为0.73和0.51.通过特征比值法对PM10中PAHs来源进行解析得出,燃煤源、柴油机动车是其主要来源。

关键词： PAHs 室内外相关性 I/O 有效透过因子来源分析

NJ-GPCA:一种面向并行空间计算的高效数据访存策略

在线全文

学校读者我要写书评

暂无评论

高技术通讯 2018年第1期28卷 1-7页

作者：姚晓邱强肖茁建方金云中国科学院计算技术研究所北京100190 中国科学院大学北京100190

针对并行矢量空间叠加分析中存在的I/O性能差及并行算法调度效率低的缺陷,提出了"去"归并通用并行计算架构(NJ-GPCA)。该架构首先基于内存数据库Redis设计内存矢量空间数据模型;其次通过数据预处理以及任务分发技术,减少进程等待,提高i/... 详细信息

针对并行矢量空间叠加分析中存在的I/O性能差及并行算法调度效率低的缺陷,提出了"去"归并通用并行计算架构(NJ-GPCA)。该架构首先基于内存数据库Redis设计内存矢量空间数据模型;其次通过数据预处理以及任务分发技术,减少进程等待,提高I/O性能;最后重新进行任务分配以及规划进程调度,避免结果数据归并收集,使得并行叠加分析算法归并收集阶段的时间复杂度由o(nlogn)降低到o(n)。实验结果表明,该方法对真实地理数据下的并行叠加分析操作,I/O时间至少减少75%,对于提高算法整体性能有明显效果。

关键词：空间叠加分析 I/O 并行计算任务调度