- 标题
- 摘要
- 关键词
- 实验方案
- 产品
-
Confidence Distance Matrix for outlier identification: A new method to improve the characterizations of surfaces measured by confocal microscopy
摘要： This paper proposes a statistical method for outlier identification for surface measurement data obtained by confocal microscopy. The implemented statistical method is Confidence Distance Matrix (CDM) which were widely used in statistics and many engineering areas, such as signal processing, sensor data fusion, information problems, etc. However, no investigations on identifying outliers in measured surface data using CDM have been found. This paper introduces and simplifies the mathematical model of CDM method. Algorithms for identifying random outliers using Monte Carlo method for uncertainty evaluation and for identifying outliers in a unique measured surface are developed and validated. For validation of the algorithms, a synthetic data SG_3-3 provided by National Institute of Standards and Technology and a data of artificial stochastic surface generated by our own algorithms are implemented. The difference of Sq of the data with outliers is 2.3342% and after deletion of outliers is 0.0037% with reference to the certified value. A type C1 spacing standard with dust dropped is measured and processed using CDM. The difference of Sa decreases from 29.65% to 3.52% after processing outliers with reference to the certified value Ra. An iron surface is measured and processed. Surface slopes and curvatures of the data in the two validations and two experiments are compared. All those parameters, the surface reconstructions, histogram of heights, and QQ plot of the measured surface data versus the data after deletion of outliers indicate our proposed method working well.
关键词： Confidence Distance matrix,outlier detection,areal surface characterization,threshold determination,Imaging Confocal Microscopy,Monte Carlo method
更新于2025-09-19 17:15:36
-
[IEEE 2019 IEEE 46th Photovoltaic Specialists Conference (PVSC) - Chicago, IL, USA (2019.6.16-2019.6.21)] 2019 IEEE 46th Photovoltaic Specialists Conference (PVSC) - Predicted Performance of High-Efficiency Photovoltaics with Energy-Selective Front Reflectors for Photon Recycling Enhancements
摘要： This paper presents a new robust EM algorithm for the finite mixture learning procedures. The proposed Spatial-EM algorithm utilizes median-based location and rank-based scatter estimators to replace sample mean and sample covariance matrix in each M step, hence enhancing stability and robustness of the algorithm. It is robust to outliers and initial values. Compared with many robust mixture learning methods, the Spatial-EM has the advantages of simplicity in implementation and statistical efficiency. We apply Spatial-EM to supervised and unsupervised learning scenarios. More specifically, robust clustering and outlier detection methods based on Spatial-EM have been proposed. We apply the outlier detection to taxonomic research on fish species novelty discovery. Two real datasets are used for clustering analysis. Compared with the regular EM and many other existing methods such as K-median, X-EM and SVM, our method demonstrates superior performance and high robustness.
关键词： EM algorithm,finite mixture,robustness,outlier detection,Clustering,spatial rank
更新于2025-09-19 17:13:59
-
[IEEE 2019 44th International Conference on Infrared, Millimeter, and Terahertz Waves (IRMMW-THz) - Paris, France (2019.9.1-2019.9.6)] 2019 44th International Conference on Infrared, Millimeter, and Terahertz Waves (IRMMW-THz) - Generation of Trains of Ultrashot Microwave Pulses by Two Coupled W-Band TWTs
摘要： This paper presents a new robust EM algorithm for the finite mixture learning procedures. The proposed Spatial-EM algorithm utilizes median-based location and rank-based scatter estimators to replace sample mean and sample covariance matrix in each M step, hence enhancing stability and robustness of the algorithm. It is robust to outliers and initial values. Compared with many robust mixture learning methods, the Spatial-EM has the advantages of simplicity in implementation and statistical efficiency. We apply Spatial-EM to supervised and unsupervised learning scenarios. More specifically, robust clustering and outlier detection methods based on Spatial-EM have been proposed. We apply the outlier detection to taxonomic research on fish species novelty discovery. Two real datasets are used for clustering analysis. Compared with the regular EM and many other existing methods such as K-median, X-EM and SVM, our method demonstrates superior performance and high robustness.
关键词： robustness,outlier detection,spatial rank,Clustering,finite mixture,EM algorithm
更新于2025-09-19 17:13:59
-
[IEEE 2019 Compound Semiconductor Week (CSW) - Nara, Japan (2019.5.19-2019.5.23)] 2019 Compound Semiconductor Week (CSW) - Ultra-low Noise Widely-Tunable Semiconductor Lasers Fully Integrated on Silicon
摘要： This paper presents a new robust EM algorithm for the ?nite mixture learning procedures. The proposed Spatial-EM algorithm utilizes median-based location and rank-based scatter estimators to replace sample mean and sample covariance matrix in each M step, hence enhancing stability and robustness of the algorithm. It is robust to outliers and initial values. Compared with many robust mixture learning methods, the Spatial-EM has the advantages of simplicity in implementation and statistical ef?ciency. We apply Spatial-EM to supervised and unsupervised learning scenarios. More speci?cally, robust clustering and outlier detection methods based on Spatial-EM have been proposed. We apply the outlier detection to taxonomic research on ?sh species novelty discovery. Two real datasets are used for clustering analysis. Compared with the regular EM and many other existing methods such as K-median, X-EM and SVM, our method demonstrates superior performance and high robustness.
关键词： ?nite mixture,spatial rank,robustness,EM algorithm,outlier detection,Clustering
更新于2025-09-19 17:13:59