- 标题
- 摘要
- 关键词
- 实验方案
- 产品
-
[IEEE 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) - Honolulu, HI (2018.7.18-2018.7.21)] 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) - Remote Heart Rate Measurement from RGB-NIR Video Based on Spatial and Spectral Face Patch Selection
摘要： In this paper, we propose a novel heart rate (HR) estimation method using simultaneously recorded RGB and near-infrared (NIR) face videos. The key idea of our method is to automatically select suitable face patches for HR estimation in both spatial and spectral domains. The spatial and spectral face patch selection enables us to robustly estimate HR under various situations, including scenes under which existing RGB camera-based methods fail to accurately estimate HR. For a challenging scene in low light and with light fluctuations, our method can successfully estimate HR for all 20 subjects (±3 beats per minute), while the RGB camera-based methods succeed only for 25% of the subjects.
关键词： spectral domain,remote sensing,spatial domain,face patch selection,heart rate estimation,RGB-NIR video
更新于2025-09-23 15:23:52
-
SCN: Switchable Context Network for Semantic Segmentation of RGB-D Images
摘要： Context representations have been widely used to profit semantic image segmentation. The emergence of depth data provides additional information to construct more discriminating context representations. Depth data preserves the geometric relationship of objects in a scene, which is generally hard to be inferred from RGB images. While deep convolutional neural networks (CNNs) have been successful in solving semantic segmentation, we encounter the problem of optimizing CNN training for the informative context using depth data to enhance the segmentation accuracy. In this paper, we present a novel switchable context network (SCN) to facilitate semantic segmentation of RGB-D images. Depth data is used to identify objects existing in multiple image regions. The network analyzes the information in the image regions to identify different characteristics, which are then used selectively through switching network branches. With the content extracted from the inherent image structure, we are able to generate effective context representations that are aware of both image structures and object relationships, leading to a more coherent learning of semantic segmentation network. We demonstrate that our SCN outperforms state-of-the-art methods on two public datasets.
关键词： Context representation,convolutional neural network (CNN),RGB-D images,semantic segmentation
更新于2025-09-23 15:23:52
-
[IEEE 2018 IEEE 2nd Colombian Conference on Robotics and Automation (CCRA) - Barranquilla, Colombia (2018.11.1-2018.11.3)] 2018 IEEE 2nd Colombian Conference on Robotics and Automation (CCRA) - A proposal for a SoC FPGA-based image processing in RGB-D sensors for robotics applications
摘要： The current robots follow clear, repetitive and logical instructions, but generally, they have problems in managing unstructured environments and reacting dynamically to these. Thus, modern robots require improved vision systems capable of obtaining information about such environments at a high acquisition rate and with high processing speeds. The growing demand for robotic platforms, both industrial and mobile, has greatly boosted the development of advanced vision systems. A weak point of traditional computer vision is that it depends on algorithms executed on a computer or server connected to the robot, often involving the need for high computing resources. Therefore, much of the efforts of the last decades have been focused on the improvement of those algorithms. Nevertheless, when the limit of traditional software processing systems (PCs, microcontrollers and microprocessors) is reached, it is necessary to migrate to a more versatile platform -which generally leads to hardware solutions-. The HW/SW design is possible because of high-frequency bridges between the Hard-Processor System (HPS) and the FPGA. Commonly, the most demanding tasks of the image processing are made in the FPGA, whereas the HPS handles the processed data and performs the high-level control function. This work presents a proposal for HW/SW integration using a SoC FPGA, for the images processing provided by the Intel Realsense?3D camera (an RGB-D sensor). This approach seeks to enhance the streamlining and ?ltering stages to obtain faster results compared to a traditional system.
关键词： SoC FPGA,Intel RealSense R200,image processing,RGB-D
更新于2025-09-23 15:23:52
-
[IEEE 2018 25th IEEE International Conference on Image Processing (ICIP) - Athens, Greece (2018.10.7-2018.10.10)] 2018 25th IEEE International Conference on Image Processing (ICIP) - HSVCNN: CNN-Based Hyperspectral Reconstruction from RGB Videos
摘要： Hyperspectral video acquisition usually requires high complexity hardware and reconstruction algorithms. In this paper, we propose a low complexity CNN-based method for hyperspectral reconstruction from ubiquitous RGB videos, which effectively exploits the temporal redundancies within RGB videos and generates high-quality hyperspectral output. Specifically, given an RGB video, we first design an efficient motion compensation network to align the RGB frames and reduce the large motion. Then, we design a temporal-adaptive fusion network to exploit the inter-frame correlation. The fusion network has the ability to determine the optimum temporal dependency within successive frames, which further promotes the hyperspectral reconstruction fidelity. Preliminary experimental results validate the superior performance of the proposed method over previous learning-based methods. To the best of our knowledge, this is the first time that RGB videos are utilized for hyperspectral reconstruction through deep learning.
关键词： Hyperspectral reconstruction,temporal-adaptive fusion,RGB videos,motion compensation
更新于2025-09-23 15:23:52
-
[IEEE 2018 24th International Conference on Pattern Recognition (ICPR) - Beijing, China (2018.8.20-2018.8.24)] 2018 24th International Conference on Pattern Recognition (ICPR) - Accurate 3-D Reconstruction with RGB-D Cameras using Depth Map Fusion and Pose Refinement
摘要： Depth map fusion is an essential part in both stereo and RGB-D based 3-D reconstruction pipelines. Whether produced with a passive stereo reconstruction or using an active depth sensor, such as Microsoft Kinect, the depth maps have noise and may have poor initial registration. In this paper, we introduce a method which is capable of handling outliers, and especially, even significant registration errors. The proposed method first fuses a sequence of depth maps into a single non-redundant point cloud so that the redundant points are merged together by giving more weight to more certain measurements. Then, the original depth maps are re-registered to the fused point cloud to refine the original camera extrinsic parameters. The fusion is then performed again with the refined extrinsic parameters. This procedure is repeated until the result is satisfying or no significant changes happen between iterations. The method is robust to outliers and erroneous depth measurements as well as even significant depth map registration errors due to inaccurate initial camera poses.
关键词： point cloud,3-D reconstruction,RGB-D cameras,pose refinement,depth map fusion,registration errors
更新于2025-09-23 15:23:52
-
[ACM Press the 2nd International Conference - Tianjin, China (2018.09.19-2018.09.21)] Proceedings of the 2nd International Conference on Biomedical Engineering and Bioinformatics - ICBEB 2018 - 3D Human Pose Estimation from RGB+D Images with Convolutional Neural Networks
摘要： In this paper, we explore 3D human pose estimation on the RGB+D images. While many researchers try to directly predict 3D pose from single RGB image, we propose a simple framework that could predict 3D pose predictions with the RGB image and depth image. Our approach is based on two aspects. On the one hand, we predicted accurate 2D joint locations from RGB image by applying the stacked hourglass networks based on the improved residual architecture. On the other hand, in view of obtained 2D joint locations, we could estimate 3D pose with the depth after calculating depth image patches. In general, compared with the state-of-the-art approaches, our model achieves signification improvement on benchmark dataset.
关键词： Deep Learning,Human Pose Estimation,RGB+D Images
更新于2025-09-23 15:23:52
-
Comparing RGB-D Sensors for Close Range Outdoor Agricultural Phenotyping
摘要： Phenotyping is the task of measuring plant attributes for analyzing the current state of the plant. In agriculture, phenotyping can be used to make decisions concerning the management of crops, such as the watering policy, or whether to spray for a certain pest. Currently, large scale phenotyping in fields is typically done using manual labor, which is a costly, low throughput process. Researchers often advocate the use of automated systems for phenotyping, relying on the use of sensors for making measurements. The recent rise of low cost, yet reasonably accurate, RGB-D sensors has opened the way for using these sensors in field phenotyping applications. In this paper, we investigate the applicability of four different RGB-D sensors for this task. We conduct an outdoor experiment, measuring plant attribute in various distances and light conditions. Our results show that modern RGB-D sensors, in particular, the Intel D435 sensor, provides a viable tool for close range phenotyping tasks in fields.
关键词： INTEL D-435,RGB-D sensors,sensors in agriculture,INTEL SR300,empirical analysis,Microsoft Kinect,phenotyping,ORBBEC ASTRA S
更新于2025-09-23 15:22:29
-
[IEEE 2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV) - Singapore, Singapore (2018.11.18-2018.11.21)] 2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV) - Bi-Manual Articulated Robot Teleoperation using an External RGB-D Range Sensor
摘要： In this paper, we present an implementation of a bi-manual teleoperation system, controlled by a human through three-dimensional (3D) skeleton extraction. The input data is given from a cheap RGB-D range sensor, such as the ASUS Xtion PRO. To achieve this, we have implemented a 3D version of the impressive OpenPose package, which was recently developed. The first stage of our method contains the execution of the OpenPose Convolutional Neural Network (CNN), using a sequence of RGB images as input. The extracted human skeleton pose localisation in two-dimensions (2D) is followed by the mapping of the extracted joint location estimations into their 3D pose in the camera frame. The output of this process is then used as input to drive the end-pose of the robotic hands relative to the human hand movements, through a whole-body inverse kinematics process in the Cartesian space. Finally, we implement the method as a ROS wrapper package and we test it on the centaur-like CENTAURO robot. Our demonstrated task is of a box and lever manipulation in real-time, as a result of a human task demonstration.
关键词： bi-manual robot,teleoperation,skeleton extraction,CENTAURO robot,OpenPose,inverse kinematics,RGB-D sensor
更新于2025-09-23 15:22:29
-
Dynamic Non-Rigid Objects Reconstruction with a Single RGB-D Sensor
摘要： This paper deals with the 3D reconstruction problem for dynamic non-rigid objects with a single RGB-D sensor. It is a challenging task as we consider the almost inevitable accumulation error issue in some previous sequential fusion methods and also the possible failure of surface tracking in a long sequence. Therefore, we propose a global non-rigid registration framework and tackle the drifting problem via an explicit loop closure. Our novel scheme starts with a fusion step to get multiple partial scans from the input sequence, followed by a pairwise non-rigid registration and loop detection step to obtain correspondences between neighboring partial pieces and those pieces that form a loop. Then, we perform a global registration procedure to align all those pieces together into a consistent canonical space as guided by those matches that we have established. Finally, our proposed model-update step helps fixing potential misalignments that still exist after the global registration. Both geometric and appearance constraints are enforced during our alignment; therefore, we are able to get the recovered model with accurate geometry as well as high fidelity color maps for the mesh. Experiments on both synthetic and various real datasets have demonstrated the capability of our approach to reconstruct complete and watertight deformable objects.
关键词： 3D reconstruction,non-rigid reconstruction,RGB-D sensor
更新于2025-09-23 15:22:29
-
The Optical Barcode Detection and Recognition Method Based on Visible Light Communication Using Machine Learning
摘要： Visible light communication (VLC) has developed rapidly in recent years. VLC has the advantages of high confidentiality, low cost, etc. It could be an effective way to connect online to offline (O2O). In this paper, an RGB-LED-ID detection and recognition method based on VLC using machine learning is proposed. Different from traditional encoding and decoding VLC, we develop a new VLC system with a form of modulation and recognition. We create different features for different LEDs to make it an Optical Barcode (OBC) based on a Complementary Metal-Oxide-Semiconductor (CMOS) senor and a pulse-width modulation (PWM) method. The features are extracted using image processing and then support vector machine (SVM) and artificial neural networks (ANN) are introduced into the scheme, which are employed as a classifier. The experimental results show that the proposed method can provide a huge number of unique LED-IDs with a high LED-ID recognition rate and its performance in dark and distant conditions is significantly better than traditional Quick Response (QR) codes. This is the first time the VLC is used in the field of Internet of Things (IoT) and it is an innovative application of RGB-LED to create features. Furthermore, with the development of camera technology, the number of unique LED-IDs and the maximum identifiable distance would increase. Therefore, this scheme can be used as an effective complement to QR codes in the future.
关键词： CMOS image sensor,machine learning,image processing,RGB-LED,visible light communication (VLC)
更新于2025-09-23 15:22:29