检索结果-南通市图书馆

outlier detection of Mixed Data Based on Neighborhood Combinatorial Entropy

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Computers, Materials & Continua 2021年第11期69卷 1765-1781页

作者： Lina Wang Qixiang Zhang Xiling Niu Yongjun Ren Jinyue Xia School of Artificial Intelligence Nanjing University of Information Science and TechnologyNanjing210044China Southern Marine Science and Engineering Guangdong Laboratory(Zhuhai) Zhuhai519080China School of Computer and Software Nanjing University of Information Science and TechnologyNanjing210044China International Business Machines Corporation(IBM) New York10504USA

outlier detection is a key research area in data mining technologies,as outlier detection can identify data inconsistent within a data set.outlier detection aims to find an abnormal data size from a large data size and has been applied in many fields including fraud detection,network intrusion detection,disaster prediction,medical diagnosis,public security,and image processing.While outlier detection has been widely applied in real systems,its effectiveness is challenged by higher dimensions and redundant data attributes,leading to detection errors and complicated calculations.The prevalence of mixed data is a current issue for outlier detection algorithms.An outlier detection method of mixed data based on neighborhood combinatorial entropy is studied to improve outlier detection performance by reducing data dimension using an attribute reduction algorithm.The significance of attributes is determined,and fewer influencing attributes are removed based on neighborhood combinatorial entropy.outlier detection is conducted using the algorithm of local outlier factor.The proposed outlier detection method can be applied effectively in numerical and mixed multidimensional data using neighborhood combinatorial entropy.In the experimental part of this paper,we give a comparison on outlier detection before and after attribute reduction.In a comparative analysis,we give results of the enhanced outlier detection accuracy by removing the fewer influencing attributes in numerical and mixed multidimensional data.

关键词： Neighborhood combinatorial entropy attribute reduction mixed data outlier detection

outlier detection over Sliding Windows for Probabilistic Data Streams

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Journal of Computer Science & Technology 2010年第3期25卷 389-400页

作者：王斌杨晓春王国仁于戈 School of Information Science and Engineering Northeastern University(Shenyang) Key Laboratory of Medical Image Computing (Northeastern University) Ministry of Education

outlier detection is a very useful technique in many applications, where data is generally uncertain and could be described using probability. While having been studied intensively in the field of deterministic data, outlier detection is still novel in the emerging uncertain data field. In this paper, we study the semantic of outlier detection on probabilistic data stream and present a new definition of distance-based outlier over sliding window. We then show the problem of detecting an outlier over a set of possible world instances is equivalent to the problem of finding the k-th element in its neighborhood. Based on this observation, a dynamic programming algorithm （DPA） is proposed to reduce the detection cost from 0（2IR（~＇d）l） to O（Ik.R（e, d）l）, where R（e, d） is the d-neighborhood of e. Furthermore, we propose a pruning-based approach （PBA） to effectively and efficiently filter non-outliers on single window, and dynamically detect recent m elements incrementally. Finally, detailed analysis and thorough experimental results demonstrate the efficiency and scalability of our approach.

关键词： outlier detection uncertain data probabilistic data stream sliding window

outlier detection and Forecasting for Bridge Health Monitoring Based on Time Series Intervention Analysis

在线全文

学校读者我要写书评

暂无评论

Structural Durability & Health Monitoring 2022年第4期16卷 323-341页

作者： Bing Qu Ping Liao Yaolong Huang College of Civil Engineering Putian UniversityPutian351100China

The method of time series analysis,applied by establishing appropriate mathematical models for bridge health monitoring data and making forecasts of structural future behavior,stands out as a novel and viable research direction for bridge state assessment.However,outliers inevitably exist in the monitoring data due to various interventions,which reduce the precision of model fitting and affect the forecasting results.Therefore,the identification of outliers is crucial for the accurate interpretation of the monitoring data.In this study,a time series model combined with outlier information for bridge health monitoring is established using intervention analysis theory,and the forecasting of the structural responses is carried out.There are three techniques that we focus on:(1)the modeling of seasonal autoregressive integrated moving average(SARIMA)model;(2)the methodology for outlier identification and amendment under the circumstances that the occurrence time and type of outliers are known and unknown;(3)forecasting of the model with outlier effects.The method was tested with a case study using monitoring data on a real bridge.The establishment of the original SARIMA model without considering outliers is first discussed,including the stationarity,order determination,parameter estimation and diagnostic checking of the model.Then the time-by-time iterative procedure for outlier detection,which is implemented by appropriate test statistics of the residuals,is performed.The SARIMA-outlier model is subsequently built.Finally,a comparative analysis of the forecasting performance between the original model and SARIMA-outlier model is carried out.The results demonstrate that proper time series models are effective in mining the characteristic law of bridge monitoring data.When the influence of outliers is taken into account,the fitted precision of the model is significantly improved and the accuracy and the reliability of the forecast are strengthened.

关键词： Structural health monitoring time series analysis outlier detection bridge state assessment bridge sensor data stress forecasting

outlier detection Algorithm Based on Iterative Clustering

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Journal of Donghua University(English Edition) 2015年第4期32卷 554-558页

作者：古平罗辛杨瑞龙张程 Department of Computer Science and Technology Chongqing University

A novel approach for outlier detection with iterative clustering( ICOD) in diverse subspaces is proposed. The proposed methodology comprises two phases,iterative clustering and outlier factor computation. During the clustering phase, multiple clusterings are detected alternatively based on an optimization procedure that incorporates terms for cluster quality and novelty relative to existing solution. Once new clusters are detected,outlier factors can be estimated from a new definition for outliers( cluster based outlier), which provides importance to the local data behavior. Experiment shows that the proposed algorithm can detect outliers which exist in different clusterings effectively even in high dimensional data sets.

关键词： clustering outlier detection dimensional reduction

Color and Shape efficiency for outlier detection from automated to user evaluation

在线全文

学校读者我要写书评

暂无评论

Visual Informatics 2022年第2期6卷 25-40页

作者： Loann Giovannangeli Romain Bourqui Romain Giot David Auber Univ.Bordeaux CNRSBordeaux INPINRIALaBRIUMR 5800F-33400 TalenceFrance

The design of efficient representations is well established as a fruitful way to explore and analyze complex or large data.In these representations,data are encoded with various visual attributes depending on the needs of the representation itself.To make coherent design choices about visual attributes,the visual search field proposes guidelines based on the human brain’s perception of features.However,information visualization representations frequently need to depict more data than the amount these guidelines have been validated on.Since,the information visualization community has extended these guidelines to a wider parameter space.This paper contributes to this theme by extending visual search theories to an information visualization context.We consider a visual search task where subjects are asked to find an unknown outlier in a grid of randomly laid out distractors.Stimuli are defined by color and shape features for the purpose of visually encoding categorical data.The experimental protocol is made of a parameters space reduction step(i.e.,sub-sampling)based on a machine learning model,and a user evaluation to validate hypotheses and measure capacity limits.The results show that the major difficulty factor is the number of visual attributes that are used to encode the outlier.When redundantly encoded,the display heterogeneity has no effect on the task.When encoded with one attribute,the difficulty depends on that attribute heterogeneity until its capacity limit(7 for color,5 for shape)is reached.Finally,when encoded with two attributes simultaneously,performances drop drastically even with minor heterogeneity.

关键词： Visual search outlier detection User evaluation Deep learning Automated evaluation

CLOF Based outlier detection Algorithm of Temperature Data for Ethylene Cracking Furnace

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Journal of Harbin Institute of Technology(New Series) 2023年第4期30卷 50-57页

作者： Yidan Xin Shaolin Hu Wenzhuo Chen He Song School of Automation and Information Engineering Xi’an University of TechnologyXi’an 710048China Guangdong Provincial Key Lab.of Petrochemical Equipment and Fault Diagnosis School of AutomationGuangdong University of Petrochemical TechnologyMaoming 525000GuangdongChina

The flue temperature is one of the important indicators to characterize the combustion state of an ethylene cracker furnace,the outliers of temperature data can lead to the false alarm.Conventional outlier detection algorithms such as the Isolation Forest algorithm and 3-sigma principle cannot detect the outliers accurately.In order to improve the detection accuracy and reduce the computational complexity,an outlier detection algorithm for flue temperature data based on the CLOF(Clipping Local outlier Factor,CLOF)algorithm is proposed.The algorithm preprocesses the normalized data using the cluster pruning algorithm,and realizes the high accuracy and high efficiency outlier detection in the outliers candidate set.Using the flue temperature data of an ethylene cracking furnace in a petrochemical plant,the main parameters of the CLOF algorithm are selected according to the experimental results,and the outlier detection effect of the Isolation Forest algorithm,the 3-sigma principle,the conventional LOF algorithm and the CLOF algorithm are compared and analyzed.The results show that the appropriate clipping coefficient in the CLOF algorithm can significantly improve the detection efficiency and detection accuracy.Compared with the outlier detection results of the Isolation Forest algorithm and 3-sigma principle,the accuracy of the CLOF detection results is increased,and the amount of data calculation is significantly reduced.

关键词： temperature data outlier detection ethylene cracker furnace clustering data clipping LOF

outlier Behavior detection for Indoor Environment Based on t-SNE Clustering

在线全文

学校读者我要写书评

暂无评论

Computers, Materials & Continua 2021年第9期68卷 3725-3736页

作者： Shinjin Kang Soo Kyun Kim School of Games Hongik UniversitySejong30016Korea Department of Computer Engineering Jeju National UniversityJeju63243Korea

In this study,we propose a low-cost system that can detect the space outlier utilization of residents in an indoor environment.We focus on the users’app usage to analyze unusual behavior,especially in indoor spaces.This is reflected in the behavioral analysis in that the frequency of using smartphones in personal spaces has recently increased.Our system facilitates autonomous data collection from mobile app logs and Google app servers and generates a high-dimensional dataset that can detect outlier behaviors.The density-based spatial clustering of applications with noise(DBSCAN)algorithm was applied for effective singular movement analysis.To analyze high-level mobile phone usage,the t-distributed stochastic neighbor embedding(t-SNE)algorithm was employed.These two clustering algorithms can effectively detect outlier behaviors in terms of movement and app usage in indoor spaces.The experimental results showed that our system enables effective spatial behavioral analysis at a low cost when applied to logs collected in actual living spaces.Moreover,large volumes of data required for outlier detection can be easily acquired.The system can automatically detect the unusual behavior of a user in an indoor space.In particular,this study aims to reflect the recent trend of the increasing use of smartphones in indoor spaces to the behavioral analysis.

关键词： outlier detection trajectory clustering behavior analysis app data smartphone

Probabilistic outlier detection for sparse multivariate geotechnical site investigation data using Bayesian learning

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Geoscience Frontiers 2021年第1期12卷 425-439页

作者： Shuo Zheng Yu-Xin Zhu Dian-Qing Li Zi-Jun Cao Qin-Xuan Deng Kok-Kwang Phoon State Key Laboratory of Water Resources and Hydropower Engineering Science Institute of Engineering Risk and Disaster PreventionWuhan University299 Bayi RoadWuhan 430072China Department of Civil and Environmental Engineering National University of SingaporeBlk E1A#07-031 Engineering Drive 2Singapore 117576Singapore

Various uncertainties arising during acquisition process of geoscience data may result in anomalous data instances(i.e.,outliers)that do not conform with the expected pattern of regular data instances.With sparse multivariate data obtained from geotechnical site investigation,it is impossible to identify outliers with certainty due to the distortion of statistics of geotechnical parameters caused by outliers and their associated statistical uncertainty resulted from data sparsity.This paper develops a probabilistic outlier detection method for sparse multivariate data obtained from geotechnical site investigation.The proposed approach quantifies the outlying probability of each data instance based on Mahalanobis distance and determines outliers as those data instances with outlying probabilities greater than 0.5.It tackles the distortion issue of statistics estimated from the dataset with outliers by a re-sampling technique and accounts,rationally,for the statistical uncertainty by Bayesian machine learning.Moreover,the proposed approach also suggests an exclusive method to determine outlying components of each outlier.The proposed approach is illustrated and verified using simulated and real-life dataset.It showed that the proposed approach properly identifies outliers among sparse multivariate data and their corresponding outlying components in a probabilistic manner.It can significantly reduce the masking effect(i.e.,missing some actual outliers due to the distortion of statistics by the outliers and statistical uncertainty).It also found that outliers among sparse multivariate data instances affect significantly the construction of multivariate distribution of geotechnical parameters for uncertainty quantification.This emphasizes the necessity of data cleaning process(e.g.,outlier detection)for uncertainty quantification based on geoscience data.

关键词： outlier detection Site investigation Sparse multivariate data Mahalanobis distance Resampling by half-means Bayesian machine learning

A Robust Fuzzy Time Series Forecasting Method Based on Multi-partition and outlier detection

在线全文

学校读者我要写书评

暂无评论

Chinese Journal of Electronics 2019年第5期28卷 899-905页

作者： QU Hua ZHANG Yanpeng LIU Wei ZHAO Jihong School of Software Engineering Xi'an Jiaotong University School of Electronics and Information Engineering Xi'an Jiaotong University School of Communication and Information Engineering Xi'an University of Posts and Telecommunication

We propose a robust fuzzy time series forecasting method based on multi-partition approach and outlier detection for forecasting market prices. The multipartition approach employs a specific partition criterion for each dimension of the time series. We use a Gaussian kernel version fuzzy C-means clustering to construct the fuzzy logic relationships and detect the outliers by calculating the grade of membership. We apply an additional model, which is trained on the set of outliers by Levenberg-Marquardt algorithm, for forecasting the outliers in testing set. The experiment results show that the proposed method improves the robustness and the average forecasting accuracy rate.

关键词： Fuzzy time series Multi-partition approach outlier detection