检索结果-南通市图书馆

data cleaning method for the process of acid production with flue gas based on improved random forest

Chinese Journal of Chemical Engineering 2023年第7期59卷 72-84页

作者： Xiaoli Li Minghua Liu Kang Wang Zhiqiang Liu Guihai Li Faculty of Information Technology Beijing University of TechnologyBeijing 100124China Beijing Key Laboratory of Computational Intelligence and Intelligent System Engineering Research Center of Digital CommunityMinistry of EducationBeijing 100124China Guixi Smelter Jiangxi Copper Co.LtdGuixi 335400China Beijing RTlink Technology Co. LtdBeijing 100102China

Acid production with flue gas is a complex nonlinear process with multiple variables and strong *** operation data is an important basis for state monitoring,optimal control,and fault ***,the operating environment of acid production with flue gas is complex and there is much *** data obtained by the detection equipment is seriously polluted and prone to abnormal phenomena such as data loss and ***,to solve the problem of abnormal data in the process of acid production with flue gas,a data cleaning method based on improved random forest is ***,an outlier data recognition model based on isolation forest is designed to identify and eliminate the outliers in the ***,an improved random forest regression model is *** algorithm is used to optimize the hyperparameters of the random forest regression *** the optimal parameter combination is found in the search space and the trend of data is ***,the improved random forest data cleaning method is used to compensate for the missing data after eliminating abnormal data and the data cleaning is *** show that the proposed method can accurately eliminate and compensate for the abnormal data in the process of acid production with flue *** method improves the accuracy of compensation for missing *** the data after cleaning,a more accurate model can be established,which is significant to the subsequent temperature *** conversion rate of SO_(2) can be further improved,thereby improving the yield of sulfuric acid and economic benefits.

关键词： Acid production data cleaning Isolation forest Random forest data compensation

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

data cleaning Based on Stacked Denoising Autoencoders and Multi-Sensor Collaborations

引用

Computers, Materials & Continua 2020年第5期63卷 691-703页

作者： Xiangmao Chang Yuan Qiu Shangting Su Deliang Yang Nanjing University of Aeronautics and Astronautics NanjingChina Collaborative Innovation Center of Novel Software Technology and Industrialization NanjingChina Michigan State University LansingUSA

Wireless sensor networks are increasingly used in sensitive event ***,various abnormal data generated by sensors greatly decrease the accuracy of the event *** many methods have been proposed to deal with the abnormal data,they generally detect and/or repair all abnormal data without further ***,besides the abnormal data caused by events,it is well known that sensor nodes prone to generate abnormal data due to factors such as sensor hardware drawbacks and random effects of external *** with all abnormal data without differentiate will result in false detection or missed detection of the *** this paper,we propose a data cleaning approach based on Stacked Denoising Autoencoders(SDAE)and multi-sensor *** detect all abnormal data by SDAE,then differentiate the abnormal data by multi-sensor *** abnormal data caused by events are unchanged,while the abnormal data caused by other factors are *** data based simulations show the efficiency of the proposed approach.

关键词： data cleaning wireless sensor networks stacked denoising autoencoders multi-sensor collaborations

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

A Review of data cleaning Methods for Web Information System

引用

Computers, Materials & Continua 2020年第3期62卷 1053-1075页

作者： Jinlin Wang Xing Wang Yuchen Yang Hongli Zhang Binxing Fang School of Computer Science and Technology Harbin Institute of TechnologyHarbin150006China

Web information system(WIS)is frequently-used and indispensable in daily social *** provides information services in many scenarios,such as electronic commerce,communities,and *** cleaning plays an essential role in various WIS scenarios to improve the quality of data *** this paper,we present a review of the state-of-the-art methods for data cleaning in *** to the characteristics of data cleaning,we extract the critical elements of WIS,such as interactive objects,application scenarios,and core technology,to classify the existing ***,after elaborating and analyzing each category,we summarize the descriptions and challenges of data cleaning methods with sub-elements such as data&user interaction,data quality rule,model,crowdsourcing,and privacy ***,we analyze various types of problems and provide suggestions for future research on data cleaning in WIS from the technology and interactive perspective.

关键词： data cleaning web information system data quality rule crowdsourcing privacy preservation

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

IoT data cleaning techniques: A survey

引用

Intelligent and Converged Networks 2022年第4期3卷 325-339页

作者： Xiaoou Ding Hongzhi Wang Genglong Li Haoxuan Li Yingze Li Yida Liu School of Computer Science and Technology Harbin Institute of TechnologyHarbin 150001China School of Mechatronics Engineering Harbin Institute of TechnologyHarbin 150001China

data cleaning is considered as an effective approach of improving data quality in order to help practitioners and researchers be devoted to downstream analysis and decision-making without worrying about data *** paper provides a systematic summary of the two main stages of data cleaning for Internet of Things(IoT)data with time series characteristics,including error data detection and data *** respect to error data detection techniques,it categorizes an overview of quantitative data error detection methods for detecting single-point errors,continuous errors,and multidimensional time series data errors and qualitative data error detection methods for detecting rule-violating ***,it provides a detailed description of error data repairing techniques,involving statistics-based repairing,rule-based repairing,and human-involved *** review the strengths and the limitations of the current data cleaning techniques under IoT data applications and conclude with an outlook on the future of IoT data cleaning.

关键词： Internet of Things(IoT) data quality data cleaning error detection data repairing

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Big data cleaning Based on Improved CLOF and Random Forest for Distribution Networks

引用

CSEE Journal of Power and Energy Systems 2024年第6期10卷 2528-2538页

作者： Jie Liu Yijia Cao Yong Li Yixiu Guo Wei Deng College of Electrical and Information Engineering Hunan UniversityChangsha 410082China State Grid Hunan Electric Power Company Limited Research Institute Changsha 410007China

In order to improve the data quality,the big data cleaning method for distribution networks is studied in this ***,the Local Outlier Factor(LOF)algorithm based on DBSCAN clustering is used to detect ***,due to the difficulty in determining the LOF threshold,a method of dynamically calculating the threshold based on the transformer districts and time is *** addition,the LOF algorithm combines the statistical distribution method to reduce the misjudgment *** at the diversity and complexity of data missing forms in power big data,this paper has improved the Random Forest imputation algorithm,which can be applied to various forms of missing data,especially the blocked missing data and even some completely missing horizontal or vertical *** data in this paper are from real data of 44 transformer districts of a certain 10 kV line in a distribution *** results show that outlier detection is accurate and suitable for any shape and multidimensional power big *** improved Random Forest imputation algorithm is suitable for all missing forms,with higher imputation accuracy and better model *** comparing the network loss prediction between the data using this data cleaning method and the data removing outliers and missing values,it can be found that the accuracy of network loss prediction has improved by nearly 4%using the data cleaning method identified in this ***,as the proportion of bad data increased,the difference between the prediction accuracy of cleaned data and that of uncleaned data is more significant.

关键词： data cleaning DBSCAN LOF missing data imputation outliers detection Random Forest

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

cleaning of Multi-Source Uncertain Time Series data Based on PageRank

引用

Journal of Donghua University(English Edition) 2023年第6期40卷 695-700页

作者：高嘉伟孙纪舟 Faculty of Computer and Software Engineering Huaiyin Institute of TechnologyHuai’an 223003China

There are errors in multi-source uncertain time series *** discovery methods for time series data are effective in finding more accurate values,but some have limitations in their *** tackle this challenge,we propose a new and convenient truth discovery method to handle time series data.A more accurate sample is closer to the truth and,consequently,to other accurate *** the mutual-confirm relationship between sensors is very similar to the mutual-quote relationship between web pages,we evaluate sensor reliability based on PageRank and then estimate the truth by sensor ***,this method does not rely on smoothness assumptions or prior knowledge of the ***,we validate the effectiveness and efficiency of the proposed method on real-world and synthetic data sets,respectively.

关键词： big data data cleaning time series truth discovery PageRank

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Impacts of Dirty data on Classification and Clustering Models:An Experimental Evaluation

引用

Journal of Computer Science & Technology 2021年第4期36卷 806-821页

作者： Zhi-Xin Qi Hong-Zhi Wang An-Jie Wang School of Computer Science and Technology Harbin Institute of TechnologyHarbin 150001China

data quality issues have attracted widespread attentions due to the negative impacts of dirty data on data mining and machine learning *** relationship between data quality and the accuracy of results could be applied on the selection of the appropriate model with the consideration of data quality and the determination of the data share to ***,rare research has focused on exploring such *** by this,this paper conducts an experimental comparison for the effects of missing,inconsistent,and conflicting data on classification and clustering *** the experimental results,we observe that dirty-data impacts are related to the error type,the error rate,and the data *** on the findings,we suggest users leverage our proposed metrics,sensibility and data quality inflection point,for model selection and data cleaning.

关键词： data quality classification clustering model selection data cleaning

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Intelligent data Pre-processing Model in Integrated Ocean Observing Network System

引用

Journal of Donghua University(English Edition) 2009年第5期26卷 499-502页

作者：韩华丁永生刘凤鸣 College of Information Science and Technology Donghua University Shanghai Academy of Science & Technology Engineering Research Center of Digitized Textile & Fashion Technology Ministry of EducationDonghua University School of Management and Economics Shandong Normal University

There are a number of dirty data in observation data set derived from integrated ocean observing network system. Thus, the data must be carefully and reasonably processed before they are used for forecasting or analysis. This paper proposes a data pre-processing model based on intelligent algorithms. Firstly, we introduce the integrated network platform of ocean observation. Next, the preprocessing model of data is presemed, and an imelligent cleaning model of data is proposed. Based on fuzzy clustering, the Kohonen clustering network is improved to fulfill the parallel calculation of fuzzy c-means clustering. The proposed dynamic algorithm can automatically f＇md the new clustering center with the updated sample data. The rapid and dynamic performance of the model makes it suitable for real time calculation, and the efficiency and accuracy of the model is proved by test results through observation data analysis.

关键词： integrated ocean observing network intelligentdata pre-processing data cleaning fuzzy soft clustering

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Can Automatic Classification Help to Increase Accuracy in data Collection?

引用

Journal of data and Information Science 2016年第3期1卷 42-58页

作者： Frederique Lang Diego Chavarro Yuxian Liu Science Policy Research Unit (SPRU) School of Business Management and Economics University of Sussex Tongji University Library Tongji University

Purpose： The authors aim at testing the performance of a set of machine learning algorithms that could improve the process of data cleaning when building datasets. Design/methodology/approach： The paper is centered on cleaning datasets gathered from publishers and online resources by the use of specific keywords. In this case, we analyzed data from the Web of Science. The accuracy of various forms of automatic classification was tested here in comparison with manual coding in order to determine their usefulness for data collection and cleaning. We assessed the performance of seven supervised classification algorithms （Support Vector Machine （SVM）, Scaled Linear Discriminant Analysis, Lasso and elastic-net regularized generalized linear models, Maximum Entropy, Regression Tree, Boosting, and Random Forest） and analyzed two properties： accuracy and recall. We assessed not only each algorithm individually, but also their combinations through a voting scheme. We also tested the performance of these algorithms with different sizes of training data. When assessing the performance of different combinations, we used an indicator of coverage to account for the agreement and disagreement on classification between algorithms. Findings： We found that the performance of the algorithms used vary with the size of the sample for training. However, for the classification exercise in this paper the best performing algorithms were SVM and Boosting. The combination of these two algorithms achieved a high agreement on coverage and was highly accurate. This combination performs well with a small training dataset （10%）, which may reduce the manual work needed for classification tasks. Research limitations： The dataset gathered has significantly more records related to the topic of interest compared to unrelated topics. This may affect the performance of some algorithms, especially in their identification of unrelated papers. Practical implications： Although the classification achieved by

关键词： Disambiguation Machine leaming data cleaning Classification Accuracy Recall Coverage

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

Matching dependencies： semantics and query answering

引用

Frontiers of Computer Science 2012年第3期6卷 278-292页

作者： Jaffer GARDEZI Leopoldo BERTOSSI Iluju KIRINGA School of Information Technology and Engineering University of Ottawa Ottawa ON K1N 6N5 Canada School of Computer Science Carleton University Ottawa ON KIS 5B6 Canada

Matching dependencies （MDs） are used to declaratively specify the identification （or matching） of cer- tain attribute values in pairs of database tuples when some similarity conditions on other values are satisfied. Their en- forcement can be seen as a natural generalization of entity resolution. In what we call the pure case of MD enforce- ment, an arbitrary value from the underlying data domain can be used for the value in common that is used for a match- ing. However, the overall number of changes of attribute val- ues is expected to be kept to a minimum. We investigate this case in terms of semantics and the properties of data clean- ing through the enforcement of MDs. We characterize the in- tended clean instances, and also the clean answers to queries, as those that are invariant under the cleaning process. The complexity of computing clean instances and clean query an- swering is investigated. Tractable and intractable cases de- pending on the MDs are identified and characterized.

关键词： databases data cleaning duplicate and entityresolution integrity constraints matching dependencies

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

欢迎您,

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

请选择保存的检索档案：

请选择收藏分类：

通借通还

欢迎您,

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：