A Cloud Service Architecture for Analyzing Big Monitoring Data

作     者：Samneet Singh Yan Liu 

作者机构：Department of Electrical and Computer ScienceConcordia University 

出 版 物：《Tsinghua Science and Technology》 (清华大学学报（自然科学版（英文版）)

年 卷 期：2016年第21卷第1期

页      面：55-70页

核心收录：

学科分类：08[工学] 080402[工学-测试计量技术及仪器] 0804[工学-仪器科学与技术] 

基　　金：supported by the Discovery grant No.RGPIN 2014-05254 from Natural Science&Engineering Research Council(NSERC) Canada 

主　　题：cloud computing REST API big data software architecture semantic web 

摘      要：Cloud monitoring is of a source of big data that are constantly produced from traces of infrastructures,platforms, and applications. Analysis of monitoring data delivers insights of the system＇s workload and usage pattern and ensures workloads are operating at optimum levels. The analysis process involves data query and extraction, data analysis, and result visualization. Since the volume of monitoring data is big, these operations require a scalable and reliable architecture to extract, aggregate, and analyze data in an arbitrary range of granularity. Ultimately, the results of analysis become the knowledge of the system and should be shared and communicated. This paper presents our cloud service architecture that explores a search cluster for data indexing and query. We develop REST APIs that the data can be accessed by different analysis modules. This architecture enables extensions to integrate with software frameworks of both batch processing（such as Hadoop） and stream processing（such as Spark） of big data. The analysis results are structured in Semantic Media Wiki pages in the context of the monitoring data source and the analysis process. This cloud architecture is empirically assessed to evaluate its responsiveness when processing a large set of data records under node failures.