检索结果-南通市图书馆

outlier detection based on multi-dimensional clustering and local density

在线全文

学校读者我要写书评

暂无评论

Journal of Central South University 2017年第6期24卷 1299-1306页

作者：首照宇李萌芽李思敏 School of Information and Communication Engineering Guilin University of Electronic TechnologyGuilin 541004China

outlier detection is an important task in data mining. In fact, it is difficult to find the clustering centers in some sophisticated multidimensional datasets and to measure the deviation degree of each potential outlier. In this work, an effective outlier detection method based on multi-dimensional clustering and local density(ODBMCLD) is proposed. ODBMCLD firstly identifies the center objects by the local density peak of data objects, and clusters the whole dataset based on the center objects. Then, outlier objects belonging to different clusters will be marked as candidates of abnormal data. Finally, the top N points among these abnormal candidates are chosen as final anomaly objects with high outlier factors. The feasibility and effectiveness of the method are verified by experiments.

关键词： data mining outlier detection outlier detection method based on multi-dimensional clustering and local density (ODBMCLD) algorithm deviation degree

outlier detection in near-infrared spectroscopic analysis by using Monte Carlo cross-validation

在线全文

学校读者我要写书评

暂无评论

Science China Chemistry 2008年第8期51卷 751-759页

作者： LIU ZhiChao CAI WenSheng SHAO XueGuang Research Center for Analytical Sciences College of ChemistryNankai UniversityTianjin 300071China

An outlier detection method is proposed for near-infrared spectral analysis. The underlying philosophy of the method is that,in random test(Monte Carlo) cross-validation,the probability of outliers presenting in good models with smaller prediction residual error sum of squares(PRESS) or in bad models with larger PRESS should be obviously different from normal samples. The method builds a large number of PLS models by using random test cross-validation at first,then the models are sorted by the PRESS,and at last the outliers are recognized according to the accumulative probability of each sample in the sorted models. For validation of the proposed method,four data sets,including three published data sets and a large data set of tobacco lamina,were investigated. The proposed method was proved to be highly efficient and veracious compared with the conventional leave-one-out(LOO) cross validation method.

关键词： near-infrared spectrum partial least squares(PLS) Monte Carlo cross validation outlier detection

Wind power forecasting based on outlier smooth transition autoregressive GARCH model

在线全文

学校读者我要写书评

暂无评论

Journal of Modern Power Systems and Clean Energy 2018年第3期6卷 532-539页

作者： Hao CHEN Fangxing LI Yurong WANG State Grid Jiangsu Electric Power Company Department of Electrical Engineering and Computer Science University of Tennessee School of Electrical Engineering Southeast University

The impacts of outlying shocks on wind power time series are explored by considering the outlier effect in the volatility of wind power time series. A novel short term wind power forecasting method based on outlier smooth transition autoregressive(OSTAR) structure is advanced, then, combined with the generalized autoregressive conditional heteroskedasticity(GARCH) model, the OSTAR-GARCH model is proposed for wind power forecasting. The proposed model is further generalized to be with fat-tail distribution.Consequently, the mechanisms of regimes against different magnitude of shocks are investigated owing to the outlier effect parameters in the proposed models. Furthermore, the outlier effect is depicted by news impact curve(NIC) and a novel proposed regime switching index(RSI). Case studies based on practical data validate the feasibility of the proposed wind power forecasting method. From the forecast performance comparison of the OSTAR-GARCH models, the OSTAR-GARCH model with fat-tail distribution proves to be promising for wind power forecasting.

关键词： OSTAR-GARCH model Regime switching index(RSI) outlier effect Wind power forecasting

Meta-path-based outlier detection in heterogeneous information network

在线全文

学校读者我要写书评

暂无评论

Frontiers of Computer Science 2020年第2期14卷 388-403页

作者： Lu LIU Shang WANG Key Laboratory of Symbol Computation and Knowledge Engineering of the Ministry of Education Changchun 130012China College of Software Jilin UniversityChangchun 130012China College of Computer Science and Technology Jilin UniversityChangchun 130012China College of Communication Engineering Jilin UniversityChangchun 130012China Department of Computer Science New Jersey Institute of TechnologyUniversity HeightsNewark NJ 07102USA

Mining outliers in heterogeneous networks is crucial to many applications,but challenges abound.In this paper,we focus on identifying meta-path-based outliers in heterogeneous information network(HIN),and calculate the similarity between different types of objects.We propose a meta-path-based outlier detection method(MPoutliers)in heterogeneous information network to deal with problems in one go under a unified framework.MPoutliers calculates the heterogeneous reachable probability by combining different types of objects and their relationships.It discovers the semantic information among nodes in heterogeneous networks,instead of only considering the network structure.It also computes the closeness degree between nodes with the same type,which extends the whole heterogeneous network.Moreover,each node is assigned with a reliable weighting to measure its authority degree.Substantial experiments on two real datasets(AMiner and Movies dataset)show that our proposed method is very effective and efficient for outlier detection.

关键词： data mining heterogeneous information network outlier detection short text similarity

Probabilistic Automatic outlier Detection for Surface Air Quality Measurements from the China National Environmental Monitoring Network

维普期刊数据库评论

在线全文

维普期刊数据库

学校读者我要写书评

暂无评论

Advances in Atmospheric Sciences 2018年第12期35卷 1522-1532页

作者： Huangjian WU Xiao TANG Zifa WANG Lin WU Miaomiao LU Lianfang WEI Jiang ZHU State Key Laboratory of Atmospheric Boundary Layer Physics and Atmospheric Chemistry Institute of Atmospheric PhysicsChinese Academy of SciencesBeijing 100029China International Center for Climate and Environment Sciences Institute of Atmospheric PhysicsChinese Academy of SciencesBeijing 100029China University of Chinese Academy of Science Beijing 100049China

Although quality assurance and quality control procedures are routinely applied in most air quality networks, outliers can still occur due to instrument malfunctions, the influence of harsh environments and the limitation of measuring methods. Such outliers pose challenges for data-powered applications such as data assimilation, statistical analysis of pollution characteristics and ensemble forecasting. Here, a fully automatic outlier detection method was developed based on the probability of residuals, which are the discrepancies between the observed and the estimated concentration values. The estimation can be conducted using filtering—or regressions when appropriate—to discriminate four types of outliers characterized by temporal and spatial inconsistency, instrument-induced low variances, periodic calibration exceptions, and less PM_(10) than PM_(2.5) in concentration observations, respectively. This probabilistic method was applied to detect all four types of outliers in hourly surface measurements of six pollutants(PM_(2.5), PM_(10),SO_2,NO_2,CO and O_3) from 1436 stations of the China National Environmental Monitoring Network during 2014-16. Among the measurements, 0.65%-5.68% are marked as outliers. with PM_(10) and CO more prone to outliers. Our method successfully identifies a trend of decreasing outliers from 2014 to 2016,which corresponds to known improvements in the quality assurance and quality control procedures of the China National Environmental Monitoring Network. The outliers can have a significant impact on the annual mean concentrations of PM_(2.5),with differences exceeding 10 μg m^(-3) at 66 sites.

关键词： probabilistic automatic outlier detection air quality observation low pass filter spatial regression bivariate normal distribution

An Efficient outlier Detection Approach on Weighted Data Stream Based on Minimal Rare Pattern Mining

在线全文

学校读者我要写书评

暂无评论

China Communications 2019年第10期16卷 83-99页

作者： Saihua Cai Ruizhi Sun Shangbo Hao Sicong Li Gang Yuan College of Information and Electrical Engineering China Agricultural UniversityBeijing 100083China Scientific Research Base for Integrated Technologies of Precision Agriculture (Animal Husbandry) the Ministry of AgricultureBeijing 100083China

The distance-based outlier detection method detects the implied outliers by calculating the distance of the points in the dataset, but the computational complexity is particularly high when processing multidimensional datasets. In addition, the traditional outlier detection method does not consider the frequency of subsets occurrence, thus, the detected outliers do not fit the definition of outliers (i.e., rarely appearing). The pattern mining-based outlier detection approaches have solved this problem, but the importance of each pattern is not taken into account in outlier detection process, so the detected outliers cannot truly reflect some actual situation. Aimed at these problems, a two-phase minimal weighted rare pattern mining-based outlier detection approach, called MWRPM-outlier, is proposed to effectively detect outliers on the weight data stream. In particular, a method called MWRPM is proposed in the pattern mining phase to fast mine the minimal weighted rare patterns, and then two deviation factors are defined in outlier detection phase to measure the abnormal degree of each transaction on the weight data stream. Experimental results show that the proposed MWRPM-outlier approach has excellent performance in outlier detection and MWRPM approach outperforms in weighted rare pattern mining.

关键词： outlier detection weighted data stream minimal weighted rare pattern mining deviation factors

Constructing Three-Dimension Space Graph for outlier Detection Algorithms in Data Mining

在线全文

学校读者我要写书评

暂无评论

Wuhan University Journal of Natural Sciences 2004年第5期9卷 585-589页

作者： ZHANG Jing 1,2 , SUN Zhi-hui 1 1.Department of Computer Science and Engineering, Southeast University, Nanjing 210096, Jiangsu, China 2.Department of Electricity and Information Engineering, Jiangsu University, Zhenjiang 212001, Jiangsu, China Department of Computer Science and Engineering Southeast University Nanjing Jiangsu China Department of Electricity and Information Engineering Jiangsu University Zhenjiang Jiangsu China

outlier detection has very important applied value in data mining literature. Different outlier detection algorithms based on distinct theories have different definitions and mining processes. The three-dimensional space graph for constructing applied algorithms and an improved GridOf algorithm were proposed in terms of analyzing the existing outlier detection algorithms from criterion and theory. Key words outlier - detection - three-dimensional space graph - data mining CLC number TP 311. 13 - TP 391 Foundation item: Supported by the National Natural Science Foundation of China (70371015)Biography: ZHANG Jing (1975-), female, Ph. D, lecturer, research direction: data mining and knowledge discovery.

关键词： outlier detection three-dimensional space graph data mining

Adaptive SINS/GPS outlier Detection and Accommodation Using Innovation Orthogonal

在线全文

学校读者我要写书评

暂无评论

Journal of Beijing Institute of Technology 2010年第4期19卷 427-431页

作者：左凯钟麦英刘百奇宫晓林 School of Control Science and Engineering Shandong University School of Instrument Science and Opto-Electronics Engineering Beihang University

The outlier detection and accommodation of integration navigation of strapdown inertial navigation systems and global position system（SINS/GPS） were studied.Based on analyzing the innovation orthogonal property in Kalman filter,an outlier adaptive detection approach was first presented,which included the determination of evaluation function and threshold and the logic decision of outlier occurrence.To effectively attenuate the influence on estimation accuracy,a modified Kalman filter algorithm was proposed by accommodation of the dynamic data with outlier.Results of data processing from vehicle-test SINS/GPS integration navigation show the effectiveness of the proposed method.

关键词： outlier detection accommodation Kalman filter SINS/GPS integration navigation

Variational Bayesian Based IMM Robust GPS Navigation Filter

在线全文

学校读者我要写书评

暂无评论

Computers, Materials & Continua 2022年第7期72卷 755-773页

作者： Dah-Jing Jwo Wei-Yeh Chang Department of Communications Navigation and Control EngineeringTaiwan Ocean University Keelung202301Taiwan TDK Taiwan Corp. YangmeiTaoyuan326021Taiwan

This paper investigates the navigational performance of Global Positioning System(GPS)using the variational Bayesian(VB)based robust filter with interacting multiple model(IMM)adaptation as the navigation processor.The performance of the state estimation for GPS navigation processing using the family ofKalman filter(KF)may be degraded due to the fact that in practical situations the statistics of measurement noise might change.In the proposed algorithm,the adaptivity is achieved by estimating the timevarying noise covariance matrices based onVB learning using the probabilistic approach,where in each update step,both the system state and time-varying measurement noise were recognized as random variables to be estimated.The estimation is iterated recursively at each time to approximate the real joint posterior distribution of state using the VB learning.One of the two major classical adaptive Kalman filter(AKF)approaches that have been proposed for tuning the noise covariance matrices is the multiple model adaptive estimate(MMAE).The IMM algorithm uses two or more filters to process in parallel,where each filter corresponds to a different dynamic or measurement model.The robust Huber’s M-estimation-based extended Kalman filter(HEKF)algorithm integrates both merits of the Huber M-estimation methodology and EKF.The robustness is enhanced by modifying the filter update based on Huber’s M-estimation method in the filtering framework.The proposed algorithm,referred to as the interactive multi-model based variational Bayesian HEKF(IMM-VBHEKF),provides an effective way for effectively handling the errors with time-varying and outlying property of non-Gaussian interference errors,such as the multipath effect.Illustrative examples are given to demonstrate the navigation performance enhancement in terms of adaptivity and robustness at the expense of acceptable additional execution time.

关键词： GPS variational bayesian Huber’sM-estimation interacting multiple model adaptive outlier multipath