摘要： Object recognition methods based on multi-modal data, color plus depth (RGB-D), usually treat each modality separately in feature extraction, which neglects implicit relations between two views and preserves noise from any view to the ?nal representation. To address these limitations, we propose a novel Canonical Correlation Analysis (CCA)-based multi-view Convolutional Neural Network (CNNs) framework for RGB-D object representation. The RGB and depth streams process corresponding images respectively, then are connected by CCA module leading to a common-correlated feature space. In addition, to embed CCA into deep CNNs in a supervised manner, two different schemes are explored. One considers CCA as a regularization term adding to the loss function (CCAR). However, solving CCA optimization directly is neither computationally ef?cient nor compatible with the mini-batch based stochastic optimization. Thus, we further propose an approximation method of CCA regularization (ACCAR), using the obtained CCA projection matrices to replace the weights of feature concatenation layer at regular intervals. Such a scheme enjoys bene?ts of full CCA regularization and is ef?cient by amortizing its cost over many training iterations. Experiments on benchmark RGB-D object recognition datasets have shown that the proposed methods outperform most existing methods using the very same of their network architectures.