Meteorological Sensor Data Storage Mechanism Based on TimescaleDB and Kafka
作者机构：College of Computer and Information Hohai University Nanjing 211100 China Postdoctoral Centre Nanjing Longyuan Micro-Electronic Company Nanjing 211106 China Jiangsu Water Resources Department Nanjing China
出 版 物：《国际计算机前沿大会会议论文集》 (International Conference of Pioneering Computer Scientists, Engineers and Educators（ICPCSEE）)
年 卷 期：2019年第1期
页 面：145-147页
主 题：NoSQL Meteorological sensor data Kafka TimescaleDB
摘 要：The scale of meteorological sensor data increases at TB level every week. Traditional relational database is inefficient in storing and processing such data and cannot satisfy many soft requirements. However, the heterogeneity and diversity of the numerous existing NoSQL systems impede the well-informed comparison and selection of a data store appropriate for a given application context. Implementing a meteorological sensor data storage mechanism is a key challenge. Therefore, a meteorological sensor data storage mechanism based on TimescaleDB and Kafka is proposed. In this solution, meteorological sensor data was acquired and transmitted by Kafka and was sent to TimescaleDB for storage and analysis. Based on simulated meteorological sensor dataset, it compared the solution with other NoSQL stores such as Redis, MongoDB, Cassandra, HBase and Riak TS. The experimental results show that the storage mechanism proposed is superior in the storage and processing of massive meteorological sensor data.
维普期刊数据库