检索结果-南通市图书馆

data cleaning method for the process of acid production with flue gas based on improved random forest

中国化学工程学报:英文版 2023年第7期59卷 72-84页

作者： Xiaoli Li Minghua Liu Kang Wang Zhiqiang Liu Guihai Li Faculty of Information Technology Beijing University of TechnologyBeijing 100124China Beijing Key Laboratory of Computational Intelligence and Intelligent System Engineering Research Center of Digital CommunityMinistry of EducationBeijing 100124China Guixi Smelter Jiangxi Copper Co.LtdGuixi 335400China Beijing RTlink Technology Co. LtdBeijing 100102China

Acid production with flue gas is a complex nonlinear process with multiple variables and strong coupling.The operation data is an important basis for state monitoring,optimal control,and fault diagnosis.However,the operating environment of acid production with flue gas is complex and there is much equipment.The data obtained by the detection equipment is seriously polluted and prone to abnormal phenomena such as data loss and outliers.Therefore,to solve the problem of abnormal data in the process of acid production with flue gas,a data cleaning method based on improved random forest is proposed.Firstly,an outlier data recognition model based on isolation forest is designed to identify and eliminate the outliers in the dataset.Secondly,an improved random forest regression model is established.Genetic algorithm is used to optimize the hyperparameters of the random forest regression model.Then the optimal parameter combination is found in the search space and the trend of data is predicted.Finally,the improved random forest data cleaning method is used to compensate for the missing data after eliminating abnormal data and the data cleaning is realized.Results show that the proposed method can accurately eliminate and compensate for the abnormal data in the process of acid production with flue gas.The method improves the accuracy of compensation for missing data.With the data after cleaning,a more accurate model can be established,which is significant to the subsequent temperature control.The conversion rate of SO_(2) can be further improved,thereby improving the yield of sulfuric acid and economic benefits.

关键词： Acid production data cleaning Isolation forest Random forest data compensation

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

IoT data cleaning techniques: A survey

引用

Intelligent and Converged Networks 2022年第4期3卷 325-339页

作者： Xiaoou Ding Hongzhi Wang Genglong Li Haoxuan Li Yingze Li Yida Liu School of Computer Science and Technology Harbin Institute of TechnologyHarbin 150001China School of Mechatronics Engineering Harbin Institute of TechnologyHarbin 150001China

data cleaning is considered as an effective approach of improving data quality in order to help practitioners and researchers be devoted to downstream analysis and decision-making without worrying about data trustworthiness.This paper provides a systematic summary of the two main stages of data cleaning for Internet of Things(IoT)data with time series characteristics,including error data detection and data repairing.In respect to error data detection techniques,it categorizes an overview of quantitative data error detection methods for detecting single-point errors,continuous errors,and multidimensional time series data errors and qualitative data error detection methods for detecting rule-violating errors.Besides,it provides a detailed description of error data repairing techniques,involving statistics-based repairing,rule-based repairing,and human-involved repairing.We review the strengths and the limitations of the current data cleaning techniques under IoT data applications and conclude with an outlook on the future of IoT data cleaning.

关键词： Internet of Things(IoT) data quality data cleaning error detection data repairing

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

data cleaning Based on Stacked Denoising Autoencoders and Multi-Sensor Collaborations

引用

Computers, Materials & Continua 2020年第5期63卷 691-703页

作者： Xiangmao Chang Yuan Qiu Shangting Su Deliang Yang Nanjing University of Aeronautics and Astronautics NanjingChina Collaborative Innovation Center of Novel Software Technology and Industrialization NanjingChina Michigan State University LansingUSA

Wireless sensor networks are increasingly used in sensitive event monitoring.However,various abnormal data generated by sensors greatly decrease the accuracy of the event detection.Although many methods have been proposed to deal with the abnormal data,they generally detect and/or repair all abnormal data without further differentiate.Actually,besides the abnormal data caused by events,it is well known that sensor nodes prone to generate abnormal data due to factors such as sensor hardware drawbacks and random effects of external sources.Dealing with all abnormal data without differentiate will result in false detection or missed detection of the events.In this paper,we propose a data cleaning approach based on Stacked Denoising Autoencoders(SDAE)and multi-sensor collaborations.We detect all abnormal data by SDAE,then differentiate the abnormal data by multi-sensor collaborations.The abnormal data caused by events are unchanged,while the abnormal data caused by other factors are repaired.Real data based simulations show the efficiency of the proposed approach.

关键词： data cleaning wireless sensor networks stacked denoising autoencoders multi-sensor collaborations

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

A Review of data cleaning Methods for Web Information System

引用

Computers, Materials & Continua 2020年第3期62卷 1053-1075页

作者： Jinlin Wang Xing Wang Yuchen Yang Hongli Zhang Binxing Fang School of Computer Science and Technology Harbin Institute of TechnologyHarbin150006China

Web information system(WIS)is frequently-used and indispensable in daily social life.WIS provides information services in many scenarios,such as electronic commerce,communities,and edutainment.data cleaning plays an essential role in various WIS scenarios to improve the quality of data service.In this paper,we present a review of the state-of-the-art methods for data cleaning in WIS.According to the characteristics of data cleaning,we extract the critical elements of WIS,such as interactive objects,application scenarios,and core technology,to classify the existing works.Then,after elaborating and analyzing each category,we summarize the descriptions and challenges of data cleaning methods with sub-elements such as data&user interaction,data quality rule,model,crowdsourcing,and privacy preservation.Finally,we analyze various types of problems and provide suggestions for future research on data cleaning in WIS from the technology and interactive perspective.

关键词： data cleaning web information system data quality rule crowdsourcing privacy preservation

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

cleaning of Multi-Source Uncertain Time Series data Based on PageRank

引用

Journal of Donghua University(English Edition) 2023年第6期40卷 695-700页

作者：高嘉伟孙纪舟 Faculty of Computer and Software Engineering Huaiyin Institute of TechnologyHuai’an 223003China

There are errors in multi-source uncertain time series data.Truth discovery methods for time series data are effective in finding more accurate values,but some have limitations in their usability.To tackle this challenge,we propose a new and convenient truth discovery method to handle time series data.A more accurate sample is closer to the truth and,consequently,to other accurate samples.Because the mutual-confirm relationship between sensors is very similar to the mutual-quote relationship between web pages,we evaluate sensor reliability based on PageRank and then estimate the truth by sensor reliability.Therefore,this method does not rely on smoothness assumptions or prior knowledge of the data.Finally,we validate the effectiveness and efficiency of the proposed method on real-world and synthetic data sets,respectively.

关键词： big data data cleaning time series truth discovery PageRank

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

data cleaning About Student Information Based on Massive Open Online Course System

引用

国际计算机前沿大会会议论文集 2020年第1期 33-43页

作者： Shengjun Yin Yaling Yi Hongzhi Wang Harbin Institute of Technology HarbinChina

Recently,Massive Open Online Courses(MOOCs)is a major way of online learning for millions of people around the world,which generates a large amount of data in the meantime.However,due to errors produced from collecting,system,and so on,these data have various inconsistencies and missing values.In order to support accurate analysis,this paper studies the data cleaning technology for online open curriculum system,including missing value-time filling for time series,and rulebased input error correction.The data cleaning algorithm designed in this paper is divided into six parts:pre-processing,missing data processing,format and content error processing,logical error processing,irrelevant data processing and correlation analysis.This paper designs and implements missing-value-filling algorithm based on time series in the missing data processing part.According to the large number of descriptive variables existing in the format and content error processing module,it proposed one-based and separability-based criteria Hot+J3+PCA.The online course data cleaning algorithm was analyzed in detail on algorithm design,implementation and testing.After a lot of rigorous testing,the function of each module performs normally,and the cleaning performance of the algorithm is of expectation.

关键词： MOOC data cleaning Time series Intermittent missing Dimension reduction

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Applying data cleaning in Changqing Oilfield Company’s data warehouse

Applying data cleaning in Changqing Oilfield Company’s data...

引用

2010 Second IITA International Conference on Geoscience and Remote Sensing (IITA-GRS 2010)

作者： Gao Xiang School of computer Northwestern Polytechnical University Xi’an China Wang Min School of communication Air Force Engineering University Xi’an China

In past ten years,Changqing Oilfield Company had constructed many information systems,mess of data had been collected in database. A great deal of knowledge and experience was embedded in these databases. To get the useful information,we must assure that all the data in the database are accurate,integrated,consistent and unique,so cleaning data of errors in structure and content is very important for data warehousing and integration. In this paper,we discussed the data quality problems appeared in the raw data,and the workflow of the data cleaning is described. For different type of the data,we adopted several cleaning method to process,details of the methods are also given in the paper. We had applied the data cleaning approaches in practice and achieved the ideal result.

关键词： data cleaning data integration data warehouse oilfield

来源： cnki会议评论

在线全文

cnki会议

学校读者我要写书评

暂无评论

Impacts of Dirty data on Classification and Clustering Models:An Experimental Evaluation

引用

Journal of Computer Science & Technology 2021年第4期36卷 806-821页

作者： Zhi-Xin Qi Hong-Zhi Wang An-Jie Wang School of Computer Science and Technology Harbin Institute of TechnologyHarbin 150001China

data quality issues have attracted widespread attentions due to the negative impacts of dirty data on data mining and machine learning results.The relationship between data quality and the accuracy of results could be applied on the selection of the appropriate model with the consideration of data quality and the determination of the data share to clean.However,rare research has focused on exploring such relationship.Motivated by this,this paper conducts an experimental comparison for the effects of missing,inconsistent,and conflicting data on classification and clustering models.FYom the experimental results,we observe that dirty-data impacts are related to the error type,the error rate,and the data size.Based on the findings,we suggest users leverage our proposed metrics,sensibility and data quality inflection point,for model selection and data cleaning.

关键词： data quality classification clustering model selection data cleaning

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Developing an Integrated IoT Cloud Based Predictive Conservation Model for Asset Management in Industry 4.0

引用

Journal of Social Computing 2023年第2期4卷 139-149页

作者： Karnam Shanmugam Kachhti Satyam Thimma Reddy Sreenivasula Reddy the Department of Computer Applications Annamacharya Institute of Technology and SciencesKarakambadiTirupathi 517520India the Department of Computer Science and Engineering Annamacharya Institute of Technology and SciencesKarakambadiTirupathi 517520India

With the advent of Industry 4.0(I4.0),predictive maintenance(PdM)methods have been widely adopted by businesses to deal with the condition of their machinery.With the help of I4.0,digital transformation,information techniques,computerised control,and communication networks,large amounts of data on operational and process conditions can be collected from multiple pieces of equipment and used to make an automated fault detection and diagnosis,all with the goal of reducing unscheduled maintenance,improving component utilisation,and lengthening the lifespan of the equipment.In this paper,we use smart approaches to create a PdM planning model.The five key steps of the created approach are as follows:(1)cleaning the data,(2)normalising the data,(3)selecting the best features,(4)making a decision about the prediction network,and(5)producing a prediction.At the outset,PdM-related data undergo data cleaning and normalisation to get everything in order and within some kind of bounds.The next step is to execute optimal feature selection in order to eliminate unnecessary data.This research presents the golden search optimization(GSO)algorithm,a powerful population-based optimization technique for efficient feature selection.The first phase of GSO is to produce a set of possible solutions or objects at random.These objects will then interact with one another using a straightforward mathematical model to find the best feasible answer.Due to the wide range over which the prediction values fall,machine learning and deep learning confront challenges in providing reliable predictions.This is why we recommend a multilayer hybrid convolution neural network(MLH-CNN).While conceptually similar to VGGNet,this approach uses fewer parameters while maintaining or improving classification correctness by adjusting the amount of network modules and channels.The projected perfect is evaluated on two datasets to show that it can accurately predict the future state of components for upkeep preparation.

关键词： Industry 4.0 predictive maintenance golden search optimization multilayer hybrid convolution neural network data cleaning

来源：

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Can Automatic Classification Help to Increase Accuracy in data Collection?

引用

Journal of data and Information Science 2016年第3期1卷 42-58页

作者： Frederique Lang Diego Chavarro Yuxian Liu Science Policy Research Unit (SPRU) School of Business Management and Economics University of Sussex Tongji University Library Tongji University

Purpose： The authors aim at testing the performance of a set of machine learning algorithms that could improve the process of data cleaning when building datasets. Design/methodology/approach： The paper is centered on cleaning datasets gathered from publishers and online resources by the use of specific keywords. In this case, we analyzed data from the Web of Science. The accuracy of various forms of automatic classification was tested here in comparison with manual coding in order to determine their usefulness for data collection and cleaning. We assessed the performance of seven supervised classification algorithms （Support Vector Machine （SVM）, Scaled Linear Discriminant Analysis, Lasso and elastic-net regularized generalized linear models, Maximum Entropy, Regression Tree, Boosting, and Random Forest） and analyzed two properties： accuracy and recall. We assessed not only each algorithm individually, but also their combinations through a voting scheme. We also tested the performance of these algorithms with different sizes of training data. When assessing the performance of different combinations, we used an indicator of coverage to account for the agreement and disagreement on classification between algorithms. Findings： We found that the performance of the algorithms used vary with the size of the sample for training. However, for the classification exercise in this paper the best performing algorithms were SVM and Boosting. The combination of these two algorithms achieved a high agreement on coverage and was highly accurate. This combination performs well with a small training dataset （10%）, which may reduce the manual work needed for classification tasks. Research limitations： The dataset gathered has significantly more records related to the topic of interest compared to unrelated topics. This may affect the performance of some algorithms, especially in their identification of unrelated papers. Practical implications： Although the classification achieved by this means is not completely accurate, the amoun

关键词： Disambiguation Machine leaming data cleaning Classification Accuracy Recall Coverage

来源：

维普期刊数据库

同方期刊数据库评论

在线全文

学校读者我要写书评

暂无评论

欢迎您,

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

请选择保存的检索档案：

请选择收藏分类：

通借通还

欢迎您,

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

在线全文

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：