- 标题
- 摘要
- 关键词
- 实验方案
- 产品
-
Survey of Object-Based Data Reduction Techniques in Observational Astronomy
摘要： Dealing with astronomical observations represents one of the most challenging areas of big data analytics. Besides huge variety of data types, dynamics related to continuous data flow from multiple sources, handling enormous volumes of data is essential. This paper provides an overview of methods aimed at reducing both the number of features/attributes as well as data instances. It concentrates on data mining approaches not related to instruments and observation tools instead working on processed object-based data. The main goal of this article is to describe existing datasets on which algorithms are frequently tested, to characterize and classify available data reduction algorithms and identify promising solutions capable of addressing present and future challenges in astronomy.
关键词： feature extraction,astronomy,dimensionality reduction,big data,data condensation
更新于2025-09-23 15:23:52
-
Analysis of trajectory similarity and configuration similarity in on-the-fly surface-hopping simulation on multi-channel nonadiabatic photoisomerization dynamics
摘要： We propose an “automatic” approach to analyze the results of the on-the-fly trajectory surface hopping simulation on the multi-channel nonadiabatic photoisomerization dynamics by considering the trajectory similarity and the configuration similarity. We choose a representative system phytochromobilin (PΦB) chromophore model to illustrate the analysis protocol. After a large number of trajectories are obtained, it is possible to define the similarity of different trajectories by the Fréchet distance and to employ the trajectory clustering analysis to divide all trajectories into several clusters. Each cluster in principle represents a photoinduced isomerization reaction channel. This idea provides an effective approach to understand the branching ratio of the multi-channel photoisomerization dynamics. For each cluster, the dimensionality reduction is employed to understand the configuration similarity in the trajectory propagation, which provides the understanding of the major geometry evolution features in each reaction channel. The results show that this analysis protocol not only assigns all trajectories into different photoisomerization reaction channels but also extracts the major molecular motion without the requirement of the pre-known knowledge of the active photoisomerization site. As a side product of this analysis tool, it is also easy to find the so-called “typical” or “representative” trajectory for each reaction channel.
关键词： trajectory similarity,multi-channel nonadiabatic photoisomerization dynamics,Fréchet distance,dimensionality reduction,phytochromobilin chromophore,on-the-fly surface-hopping simulation,configuration similarity,clustering analysis
更新于2025-09-23 15:23:52
-
[IEEE IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium - Valencia (2018.7.22-2018.7.27)] IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium - Hyper-Laplacian Regularized Low-Rank Tensor Decomposition for Hyperspectral Anomaly Detection
摘要： This paper presents a novel method for hyperspectral anomaly detection considering the spectral redundancy and exploiting spectral-spatial information at the same time. We proposed a Hyper-Laplacian regularized low-rank tensor decomposition method combing with dimensionality reduction framework. Firstly, k-means++ algorithm is implemented to spectral bands and centers of each group are selected to reduce the HSI dimensionality in spectral direction. To jointly utilize spectral-spatial information, the cubic data (two spatial dimensions and one spectral dimension) is treated as a 3-order tensor. Then the non-local self-similarity is fully explored in our method. For the reason to reduce the ringing artifacts caused by over-lapped segmentation in exploring the non-local self-similarity, we introduce the hyper-Laplacian constrained low-rank tensor decomposition and we get the separated background and residual parts. Finally, to eliminate the effect of Gaussian noise, we use local-RX basic detector to detect the residual matrix. Experimental results on two real hyperspectral data sets verified the effectiveness of the proposed algorithms for HSI anomaly detection.
关键词： low-rank tensor decomposition,hyperspectral anomaly detection,Dimensionality reduction
更新于2025-09-23 15:23:52
-
Learning compact q-space representations for multi-shell diffusion-weighted MRI
摘要： Diffusion-weighted MRI measures the direction and scale of the local diffusion process in every voxel through its spectrum in q-space, typically acquired in one or more shells. Recent developments in microstructure imaging and multi-tissue decomposition have sparked renewed attention in the radial b-value dependence of the signal. Applications in motion correction and outlier rejection therefore require a compact linear signal representation that extends over the radial as well as angular domain. Here, we introduce SHARD, a data-driven representation of the q-space signal based on spherical harmonics and a radial decomposition into orthonormal components. This representation provides a complete, orthogonal signal basis, tailored to the spherical geometry of q-space and calibrated to the data at hand. We demonstrate that the rank-reduced decomposition outperforms model-based alternatives in human brain data, whilst faithfully capturing the micro- and meso-structural information in the signal. Furthermore, we validate the potential of joint radial-spherical as compared to single-shell representations. As such, SHARD is optimally suited for applications that require low-rank signal predictions, such as motion correction and outlier rejection. Finally, we illustrate its application for the latter using outlier robust regression.
关键词： Diffusion-weighted imaging,Blind source separation,Multi-shell HARDI,Dimensionality reduction
更新于2025-09-23 15:23:52
-
Laplacian Regularized Spatial-Aware Collaborative Graph for Discriminant Analysis of Hyperspectral Imagery
摘要： Dimensionality Reduction (DR) models are of signi?cance to extract low-dimensional features for Hyperspectral Images (HSIs) data analysis where there exist lots of noisy and redundant spectral features. Among many DR techniques, the Graph-Embedding Discriminant Analysis framework has demonstrated its effectiveness for HSI feature reduction. Based on this framework, many representation based models are developed to learn the similarity graphs, but most of these methods ignore the spatial information, resulting in unsatisfactory performance of DR models. In this paper, we ?rstly propose a novel supervised DR algorithm termed Spatial-aware Collaborative Graph for Discriminant Analysis (SaCGDA) by introducing a simple but ef?cient spatial constraint into Collaborative Graph-based Discriminate Analysis (CGDA) which is inspired by recently developed Spatial-aware Collaborative Representation (SaCR). In order to make the representation of samples on the data manifold smoother, i.e., similar pixels share similar representations, we further add the spectral Laplacian regularization and propose the Laplacian regularized SaCGDA (LapSaCGDA), where the two spectral and spatial constraints can exploit the intrinsic geometric structures embedded in HSIs ef?ciently. Experiments on three HSIs data sets verify that the proposed SaCGDA and LapSaCGDA outperform other state-of-the-art methods.
关键词： hyperspectral imagery,graph embedding,dimensionality reduction,collaborative representation,discriminant analysis
更新于2025-09-23 15:22:29
-
[IEEE 2019 21st International Conference on Transparent Optical Networks (ICTON) - Angers, France (2019.7.9-2019.7.13)] 2019 21st International Conference on Transparent Optical Networks (ICTON) - Surface Texturing with Multi-objective Particle Swarm Optimization for Absorption Enhancement in Silicon Photovoltaics
摘要： In large datasets, manual data verification is impossible, and we must expect the number of outliers to increase with data size. While principal component analysis (PCA) can reduce data size, and scalable solutions exist, it is well-known that outliers can arbitrarily corrupt the results. Unfortunately, state-of-the-art approaches for robust PCA are not scalable. We note that in a zero-mean dataset, each observation spans a one-dimensional subspace, giving a point on the Grassmann manifold. We show that the average subspace corresponds to the leading principal component for Gaussian data. We provide a simple algorithm for computing this Grassmann Average (GA), and show that the subspace estimate is less sensitive to outliers than PCA for general distributions. Because averages can be efficiently computed, we immediately gain scalability. We exploit robust averaging to formulate the Robust Grassmann Average (RGA) as a form of robust PCA. The resulting Trimmed Grassmann Average (TGA) is appropriate for computer vision because it is robust to pixel outliers. The algorithm has linear computational complexity and minimal memory requirements. We demonstrate TGA for background modeling, video restoration, and shadow removal. We show scalability by performing robust PCA on the entire Star Wars IV movie; a task beyond any current method. Source code is available online.
关键词： subspace estimation,Dimensionality reduction,robust principal component analysis
更新于2025-09-23 15:19:57
-
[IEEE 2019 IEEE 46th Photovoltaic Specialists Conference (PVSC) - Chicago, IL, USA (2019.6.16-2019.6.21)] 2019 IEEE 46th Photovoltaic Specialists Conference (PVSC) - Silicon Heterojunction and Half-Cell configuration: optimization path for increased module power
摘要： In large datasets, manual data verification is impossible, and we must expect the number of outliers to increase with data size. While principal component analysis (PCA) can reduce data size, and scalable solutions exist, it is well-known that outliers can arbitrarily corrupt the results. Unfortunately, state-of-the-art approaches for robust PCA are not scalable. We note that in a zero-mean dataset, each observation spans a one-dimensional subspace, giving a point on the Grassmann manifold. We show that the average subspace corresponds to the leading principal component for Gaussian data. We provide a simple algorithm for computing this Grassmann Average (GA), and show that the subspace estimate is less sensitive to outliers than PCA for general distributions. Because averages can be efficiently computed, we immediately gain scalability. We exploit robust averaging to formulate the Robust Grassmann Average (RGA) as a form of robust PCA. The resulting Trimmed Grassmann Average (TGA) is appropriate for computer vision because it is robust to pixel outliers. The algorithm has linear computational complexity and minimal memory requirements. We demonstrate TGA for background modeling, video restoration, and shadow removal. We show scalability by performing robust PCA on the entire Star Wars IV movie; a task beyond any current method. Source code is available online.
关键词： subspace estimation,Dimensionality reduction,robust principal component analysis
更新于2025-09-23 15:19:57
-
Locally Weighted Discriminant Analysis for Hyperspectral Image Classification
摘要： A hyperspectral image (HSI) contains a great number of spectral bands for each pixel, which will limit the conventional image classification methods to distinguish land-cover types of each pixel. Dimensionality reduction is an effective way to improve the performance of classification. Linear discriminant analysis (LDA) is a popular dimensionality reduction method for HSI classification, which assumes all the samples obey the same distribution. However, different samples may have different contributions in the computation of scatter matrices. To address the problem of feature redundancy, a new supervised HSI classification method based on locally weighted discriminant analysis (LWDA) is presented. The proposed LWDA method constructs a weighted discriminant scatter matrix model and an optimal projection matrix model for each training sample, which is on the basis of discriminant information and spatial-spectral information. For each test sample, LWDA searches its nearest training sample with spatial information and then uses the corresponding projection matrix to project the test sample and all the training samples into a low-dimensional feature space. LWDA can effectively preserve the spatial-spectral local structures of the original HSI data and improve the discriminating power of the projected data for the final classification. Experimental results on two real-world HSI datasets show the effectiveness of the proposed LWDA method compared with some state-of-the-art algorithms. Especially when the data partition factor is small, i.e., 0.05, the overall accuracy obtained by LWDA increases by about 20% for Indian Pines and 17% for Kennedy Space Center (KSC) in comparison with the results obtained when directly using the original high-dimensional data.
关键词： hyperspectral image (HSI) classification,linear discriminant analysis (LDA),spatial-spectral information,dimensionality reduction
更新于2025-09-19 17:15:36
-
Loss of target information in full pixel and subpixel target detection in hyperspectral data with and without dimensionality reduction
摘要： In most hyperspectral target detection applications, targets are usually small and require both spatial as well as spectral detection. Hyperspectral imaging facilitates target detection (TD) applications greatly, however, due to large spectral content, hyperspectral data requires dimensionality reduction (DR) which also leads to loss of target information both at full pixel and subpixel level. Literature reports many DR and TD algorithms in practice. Several studies have focussed on assessing the loss of target information in DR, however, not much work seems to have been done to assess loss of target information in full pixel and subpixel TD in hyperspectral data with and without DR. This paper seeks to study various combinations of DR techniques combined with full pixel and subpixel TD algorithms. The results indicate that in the case of full pixel targets, both DR and TD contribute to the loss of target information, however, there is more loss of target information in the case when DR precedes TD in comparison to a case where TD is applied without DR. In the case of subpixel TD, however, there appears to be loss of subpixel target information in the case where TD alone is performed in comparison to a case where DR precedes TD.
关键词： Spectral unmixing,Mixed pixel,Subpixel target detection,Target information,Dimensionality reduction,Full pixel target detection
更新于2025-09-19 17:15:36
-
[IEEE 2019 IEEE Research and Applications of Photonics in Defense Conference (RAPID) - Miramar Beach, FL, USA (2019.8.19-2019.8.21)] 2019 IEEE Research and Applications of Photonics in Defense Conference (RAPID) - Invited Talk: "High Resolution Space/Time Imaging of Shockwaves Generated by Remote Laser Plasmas Produced by Light Filaments"
摘要： In this paper, we approach the problem of forecasting a time series (TS) of an electrical load measured on the Azienda Comunale Energia e Ambiente (ACEA) power grid, the company managing the electricity distribution in Rome, Italy, with an echo state network (ESN) considering two different leading times of 10 min and 1 day. We use a standard approach for predicting the load in the next 10 min, while, for a forecast horizon of one day, we represent the data with a high-dimensional multi-variate TS, where the number of variables is equivalent to the quantity of measurements registered in a day. Through the orthogonal transformation returned by PCA decomposition, we reduce the dimensionality of the TS to a lower number k of distinct variables; this allows us to cast the original prediction problem in k different one-step ahead predictions. The overall forecast can be effectively managed by k distinct prediction models, whose outputs are combined together to obtain the final result. We employ a genetic algorithm for tuning the parameters of the ESN and compare its prediction accuracy with a standard autoregressive integrated moving average model.
关键词： PCA,dimensionality reduction,electric load prediction,smart grid,genetic algorithm,forecasting,echo state network,Time-series
更新于2025-09-19 17:13:59