- 标题
- 摘要
- 关键词
- 实验方案
- 产品
-
Learning-based Computation Offloading for IoT Devices with Energy Harvesting
摘要： Internet of Things (IoT) devices can apply mobile edge computing (MEC) and energy harvesting (EH) to provide high level experiences for computational intensive applications and concurrently to prolong the lifetime of the battery. In this paper, we propose a reinforcement learning (RL) based offloading scheme for an IoT device with EH to select the edge device and the offloading rate according to the current battery level, the previous radio transmission rate to each edge device and the predicted amount of the harvested energy. This scheme enables the IoT device to optimize the offloading policy without knowledge of the MEC model, the energy consumption model and the computation latency model. Further, we present a deep RL based offloading scheme to further accelerate the learning speed. Their performance bounds in terms of the energy consumption, computation latency and utility are provided for three typical offloading scenarios and verified via simulations for an IoT device that uses wireless power transfer for energy harvesting. Simulation results show that the proposed RL based offloading scheme reduces the energy consumption, computation latency and task drop rate and thus increases the utility of the IoT device in the dynamic MEC in comparison with the benchmark offloading schemes.
关键词： Mobile edge computing,energy harvesting,reinforcement learning,computation offloading,Internet of Things
更新于2025-09-23 15:22:29
-
VLC and D2D Heterogeneous Network Optimization: A Reinforcement Learning Approach Based on Equilibrium Problems with Equilibrium Constraints
摘要： The Radio Frequency (RF) spectrum crunch has triggered the harnessing of other sources of bandwidth, for which visible light is a promising candidate. Even though Visible Light Communication (VLC) ensures high capacity, coverage is limited. This necessitates the integration of VLC and Device-to-Device (D2D) technologies into heterogeneous networks. In particular, mobile users which are accessible by the VLC transmitters can relay data to mobile users which are not, by means of D2D communication. However, due to the distributed behaviors of mobile users, determining optimal data transmission routes from VLC transmitters to end mobile devices is a major challenge. In this paper, we propose a Reinforcement Learning (RL) based approach to determine multi-hop data transmission routes in an indoor VLC-D2D heterogeneous network. We obtain the rewards for the RL based method dynamically, by formulating the interactions between the mobile users relaying the data as an Equilibrium Problem with Equilibrium Constraints (EPEC) and using Alternating Direction Method of Multipliers (ADMM) to solve it. The proposed technique can achieve optimal data transmission routes in a distributed manner. Simulation results demonstrate the effectiveness of the proposed approach, showing that transmission routes with low delays and high capacities can be achieved through the learning algorithm.
关键词： Device-to-Device,Reinforcement Learning,Visible Light Communication,Alternating Direction Method of Multipliers,Heterogeneous Network,Equilibrium Problem with Equilibrium Constraints
更新于2025-09-23 15:22:29
-
Design, Simulation and Fabrication of a High Gain Low Sidelobe Level Waveguide Slot Array Antenna at X-band with Zero Beam Tilts in Both Azimuth and Elevation Directions
摘要： This paper presents the development of an intelligent dynamic energy management system (I-DEMS) for a smart microgrid. An evolutionary adaptive dynamic programming and reinforcement learning framework is introduced for evolving the I-DEMS online. The I-DEMS is an optimal or near-optimal DEMS capable of performing grid-connected and islanded microgrid operations. The primary sources of energy are sustainable, green, and environmentally friendly renewable energy systems (RESs), e.g., wind and solar; however, these forms of energy are uncertain and nondispatchable. Backup battery energy storage and thermal generation were used to overcome these challenges. Using the I-DEMS to schedule dispatches allowed the RESs and energy storage devices to be utilized to their maximum in order to supply the critical load at all times. Based on the microgrid’s system states, the I-DEMS generates energy dispatch control signals, while a forward-looking network evaluates the dispatched control signals over time. Typical results are presented for varying generation and load profiles, and the performance of I-DEMS is compared with that of a decision tree approach-based DEMS (D-DEMS). The robust performance of the I-DEMS was illustrated by examining microgrid operations under different battery energy storage conditions.
关键词： microgrid,Adaptive dynamic programming,reinforcement learning,evolutionary computing,dynamic energy management system (DEMS),renewable energy,neural networks
更新于2025-09-23 15:21:01
-
A Scalable Soft-Switching Photovoltaic Inverter with Full-Range ZVS and Galvanic Isolation
摘要： The introduction of heterogeneous wireless mesh technologies provides an opportunity for higher network capacity, wider coverage, and higher quality of service (QoS). Each wireless device utilizes different standards, data formats, protocols, and access technologies. However, the diversity and complexity of such technologies create challenges for traditional control and management systems. This paper proposes a heterogeneous metropolitan area network architecture that combines an IEEE 802.11 wireless mesh network (WMN) with a long-term evolution (LTE) network. In addition, a new heterogeneous routing protocol and a routing algorithm based on reinforcement learning called cognitive heterogeneous routing are proposed to select the appropriate transmission technology based on parameters from each network. The proposed heterogeneous network overcomes the problems of sending packets over long paths, island nodes, and interference in WMNs and increases the overall capacity of the combined network by utilizing unlicensed frequency bands instead of buying more license frequency bands for LTE. The work is validated through extensive simulations that indicate that the proposed heterogeneous WMN outperforms the LTE and Wi-Fi networks when used individually. The simulation results show that the proposed network achieves an increase of up to 200% in throughput compared with Wi-Fi-only networks or LTE-only networks.
关键词： routing protocol,long-term evolution (LTE),reinforcement learning,next-generation network,Heterogeneous networks,wireless mesh network (WMN)
更新于2025-09-23 15:21:01
-
Cache-enabled physical-layer secure game against smart uAV-assisted attacks in b5G NOMA networks
摘要： This paper investigates cache-enabled physical-layer secure communication in a no-orthogonal multiple access (NOMA) network with two users, where an intelligent unmanned aerial vehicle (UAV) is equipped with attack module which can perform as multiple attack modes. We present a power allocation strategy to enhance the transmission security. To this end, we propose an algorithm which can adaptively control the power allocation factor for the source station in NOMA network based on reinforcement learning. The interaction between the source station and UAV is regarded as a dynamic game. In the process of the game, the source station adjusts the power allocation factor appropriately according to the current work mode of the attack module on UAV. To maximize the benefit value, the source station keeps exploring the changing radio environment until the Nash equilibrium (NE) is reached. Moreover, the proof of the NE is given to verify the strategy we proposed is optimal. Simulation results prove the effectiveness of the strategy.
关键词： UAV,Cache,B5G,Reinforcement learning,Physical-layer security,NOMA
更新于2025-09-23 15:19:57
-
A Ship Rotation Detection Model in Remote Sensing Images Based on Feature Fusion Pyramid Network and Deep Reinforcement Learning
摘要： Ship detection plays an important role in automatic remote sensing image interpretation. The scale difference, large aspect ratio of ship, complex remote sensing image background and ship dense parking scene make the detection task difficult. To handle the challenging problems above, we propose a ship rotation detection model based on a Feature Fusion Pyramid Network and deep reinforcement learning (FFPN-RL) in this paper. The detection network can efficiently generate the inclined rectangular box for ship. First, we propose the Feature Fusion Pyramid Network (FFPN) that strengthens the reuse of different scales features, and FFPN can extract the low level location and high level semantic information that has an important impact on multi-scale ship detection and precise location of dense parking ships. Second, in order to get accurate ship angle information, we apply deep reinforcement learning to the inclined ship detection task for the first time. In addition, we put forward prior policy guidance and a long-term training method to train an angle prediction agent constructed through a dueling structure Q network, which is able to iteratively and accurately obtain the ship angle. In addition, we design soft rotation non-maximum suppression to reduce the missed ship detection while suppressing the redundant detection boxes. We carry out detailed experiments on the remote sensing ship image dataset, and the experiments validate that our FFPN-RL ship detection model has efficient detection performance.
关键词： feature map fusion,deep reinforcement learning,ship detection,convolution neural network
更新于2025-09-19 17:15:36
-
[IEEE 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) - Madrid, Spain (2018.10.1-2018.10.5)] 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) - Image-Based Visual Servoing Controller for Multirotor Aerial Robots Using Deep Reinforcement Learning
摘要： In this paper, we propose a novel Image-Based Visual Servoing (IBVS) controller for multirotor aerial robots based on a recent deep reinforcement learning algorithm named Deep Deterministic Policy Gradients (DDPG). The proposed RL-IBVS controller is successfully trained in a Gazebo-based simulation scenario in order to learn the appropriate IBVS policy for directly mapping a state, based on errors in the image, to the linear velocity commands of the aerial robot. A thorough validation of the proposed controller has been conducted in simulated and real flight scenarios, demonstrating outstanding capabilities in object following applications. Moreover, we conduct a detailed comparison of the RL-IBVS controller with respect to classic and partitioned IBVS approaches.
关键词： Real Flight Experiments,Image-Based Visual Servoing,Deep Reinforcement Learning,Simulation,Aerial Robots,DDPG
更新于2025-09-19 17:15:36
-
[IEEE 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) - Madrid, Spain (2018.10.1-2018.10.5)] 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) - A Deep Reinforcement Learning Technique for Vision-Based Autonomous Multirotor Landing on a Moving Platform
摘要： Deep learning techniques for motion control have recently been qualitatively improved, since the successful application of Deep Q-Learning to the continuous action domain in Atari-like games. Based on these ideas, Deep Deterministic Policy Gradients (DDPG) algorithm was able to provide impressive results in continuous state and action domains, which are closely linked to most of the robotics-related tasks. In this paper, a vision-based autonomous multirotor landing maneuver on top of a moving platform is presented. The behaviour has been completely learned in simulation without prior human knowledge and by means of deep reinforcement learning techniques. Since the multirotor is controlled in attitude, no high level state estimation is required. The complete behaviour has been trained with continuous action and state spaces, and has provided proper results (landing at a maximum velocity of 2 m/s). Furthermore, it has been validated in a wide variety of conditions, for both simulated and real-flight scenarios, using a low-cost, lightweight and out-of-the-box consumer multirotor.
关键词： vision-based control,deep reinforcement learning,multirotor,moving platform,autonomous landing
更新于2025-09-19 17:15:36
-
[IEEE 2019 23rd International Conference on Mechatronics Technology (ICMT) - SALERNO, Italy (2019.10.23-2019.10.26)] 2019 23rd International Conference on Mechatronics Technology (ICMT) - Toward the Application of Reinforcement Learning to the Intensity Control of a Seeded Free-Electron Laser
摘要： The optimization of particle accelerators is a challenging task, and many different approaches have been proposed in years, to obtain an optimal tuning of the plant and to keep it optimally tuned despite drifts or disturbances. Indeed, the classical model-free approaches (such as Gradient Ascent or Extremum Seeking algorithms) have intrinsic limitations. To overcome those limitations, Machine Learning techniques, in particular, the Reinforcement Learning, are attracting more and more attention in the particle accelerator community. The purpose of this paper is to apply a Reinforcement Learning model-free approach to the alignment of a seed laser, based on a rather general target function depending on the laser trajectory. The study focuses on the alignment of the lasers at FERMI, the free-electron laser facility at Elettra Sincrotrone Trieste. In particular, we employ Q-learning with linear function approximation and report experimental results obtained in two setups, which are the actual setups where the final application has to be deployed. Despite the simplicity of the approach, we report satisfactory preliminary results, that represent the first step toward a fully automatic procedure for seed laser to the electron beam. Such a superimposition is, at present, performed manually.
关键词： Free-Electron Laser,Particle Accelerators,Q-learning,Reinforcement Learning,Machine Learning
更新于2025-09-19 17:13:59
-
[IEEE 2019 21st International Conference on Transparent Optical Networks (ICTON) - Angers, France (2019.7.9-2019.7.13)] 2019 21st International Conference on Transparent Optical Networks (ICTON) - Dynamic Control of Transparent Optical Networks with Adaptive State-Value Assessment Enabled by Reinforcement Learning
摘要： For efficient and dynamic path operations in transparent optical networks, routing and wavelength assignment (RWA) must be optimized in terms of not only link-resource utilization but also traffic distribution. In this paper, we propose a reinforcement-learning-based RWA algorithm that maximizes the number of paths to be accommodated to a network with pre-training using estimated traffic distributions. Numerical experiments elucidate that the number of paths accommodated increases by up to 9.1%.
关键词： transparent optical network,network-state value,reinforcement learning,routing and wavelength assignment,dynamic network control
更新于2025-09-19 17:13:59