检索结果-南通市图书馆

A Task Allocation Method for Stream Processing with Recovery Latency Constraint

Journal of Computer Science & Technology 2018年第6期33卷 1125-1139页

作者： Hong-Liang Li Jie Wu Zhen Jiang Xiang Li Xiao-Hui Wei College of Computer Science and Technology Jilin University Changchun 130012 China Key Laboratory of Symbolic Computation and Knowledge Engineering of the Ministry of Education Changchun 130012 China Department of Computer and Information Sciences Temple University Philadelphia PA 19122 U.S.A. Department of Computer Science West Chester University of Pennsylvania West Chester PA 19383 U.S.A.

Stream processing applications continuously process large amounts of online streaming data in real time or nearreal time. They have strict latency constraints. However, the continuous processing makes them vulnerable to any failures,and the recoveries may slow down the entire processing pipeline and break latency constraints. The upstream backupscheme is one of the most widely applied fault-tolerant schemes for stream processing systems. It introduces complexbackup dependencies to tasks, which increases the difficulty of controlling recovery latencies. Moreover, when dependenttasks are located on the same processor, they fail at the same time in processor-level failures, bringing extra recovery latencies that increase the impacts of failures. This paper studies the relationship between the task allocation and therecovery latency of a stream processing application. We present a correlated failure effect model to describe the recoverylatency of a stream topology in processor-level failures under a task allocation plan. We introduce a recovery-latency awaretask allocation problem （RTAP） that seeks task allocation plans for stream topologies that will achieve guaranteed recoverylatencies. We discuss the difference between RTAP and classic task allocation problems and present a heuristic algorithmwith a computational complexity of O（n log2 n） to solve the problem. Extensive experiments were conducted to verify thecorrectness and effectiveness of our approach. It improves the resource usage by 15%-20% on average.

关键词： stream processing task allocation fault-tolerance upstream backup, recovery latency

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

欢迎您,

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

请选择保存的检索档案：

请选择收藏分类：

通借通还

欢迎您,

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：