[1]王 峰,马星宇,孟鹏帅,等.基于深度强化学习的无人机边缘计算任务卸载策略[J].郑州大学学报(工学版),2024,45(pre1):10.[doi:10.13705/j.issn.1671-6833.2025.01.018]
　WANG Feng,MA Xingyu,MENG Pengshuai,et al.Task Offloading Strategy of UAV Edge Computing Based on Deep Reinforcement Learning[J].Journal of Zhengzhou University (Engineering Science),2024,45(pre1):10.[doi:10.13705/j.issn.1671-6833.2025.01.018]
点击复制

基于深度强化学习的无人机边缘计算任务卸载策略()
分享到：

《郑州大学学报(工学版)》[ISSN:1671-6833/CN:41-1339/T]

卷:
45
期数:
2024年pre1
页码:
10
栏目:
出版日期:
2025-01-30

文章信息/Info

Title:
Task Offloading Strategy of UAV Edge Computing Based on Deep Reinforcement Learning
作者:
王 峰1; 马星宇2; 孟鹏帅2; 赵 薇2; 翟伟光2
1.太原理工大学 电气与动力工程学院，山西 太原030024；2. 太原理工大学 电子信息与光学工程学院，山西 太原030600
Author(s):
WANG Feng;  MA Xingyu;  MENG Pengshuai;  ZHAO Wei;  ZHAI Weiguang
关键词:
无人机; 边缘计算; 任务卸载; 深度强化学习; 资源分配
Keywords:
UAV;  Edge computing;  Task unloading;  Deep reinforcement learning;  Resource allocation
分类号:
TN929. 5TP391. 9
DOI:
10.13705/j.issn.1671-6833.2025.01.018
文献标志码:
A
摘要:

针对地理条件较为复杂的环境中存在的缺乏基础设施 、 任务延时高和带宽需求量大等问题 , 提出一种联合任务卸载和功率分配的多级移动边缘计算 ( MEC ) 系统模型 。 所提模型考虑将配备 MEC 的服务器部署在无人机附近提供计算服务 , 综合分析无人机的任务卸载 、 功耗和计算资源分配等问题并给出度量方法 , 同时考虑无人机可执行的任务类型以及任务对无人机的 CPU 和 GPU 要求 , 将该问题表述为混合整数非线性问题 。针对该问题提出一种基于深度强化学习的计算任务卸载算法 , 该算法基于改进双深度 Q 学习算法 , 在深度强化学习中利用深度神经网络找到无人机之间的映射 , 从状态空间中找到潜在的模式并估计最优动作 , 并使用无模型的 DRL 方法 , 使每个无人机根据局部观察快速做出卸载决策 。仿真结果表明 : 所提算法相比 LCGP 算法 , 平均卸载成本降低了42. 8% ; 相比 DDPG 算法 , 能耗减少了 16% ; 相比 DDQN 算法 , 任务执行延迟减少了12. 9% 。


Abstract:
Aiming at the problems such as lack of infrastructure, high task delay and high bandwidth demand in complex geographical conditions, this paper proposes a multi-stage mobile edge computing system model which combines computing offload and power distribution. In this model, a server equipped with MEC is deployed near the UAV to provide computing services, and the problems such as task offloading, power consumption and computing resource allocation of the UAV are comprehensively analyzed and the measurement methods are given. At the same time, the types of tasks that the UAV can perform and the requirements of the CPU and GPU on the UAV are considered. The problem is expressed as a mixed integer nonlinear problem. Secondly, a task computing offloading algorithm based on deep reinforcement learning is proposed to solve this problem. Based on the improved double deep Q learning algorithm, the algorithm uses deep neural network to find the mapping between drones in deep reinforcement learning, find potential patterns from the state space and estimate the optimal action, and uses model-free DRL method. Enable each drone to make quick unloading decisions based on local observations. In order to verify the effectiveness of the proposed scheme, detailed simulation is carried out in this paper. Simulation results show that the proposed algorithm reduces the average unloading cost by 42.8% compared with LCGP algorithm. Compared with DDPG algorithm, the energy consumption is reduced by 16%. Compared with DDQN algorithm, the task execution delay is reduced by 12.9%

参考文献/References:

[ 1 ] PANWAR P, SHABAZ M, NAZIR S, et al. Generic edge computing system for optimization and computation offloading of unmanned aerial vehicle [ J ] . Computers and Electrical Engineering, 2023, 109: 108779.

[ 2 ]陈新颖, 盛敏, 李博, 等. 面向 6G 的无人机通信综述[ J] . 电子与信息学报, 2022, 44(3) : 781-789.

CHEN X Y, SHENG M, LI B, et al. Survey on unmanned aerial vehicle communications for 6G[ J] . Journal of Electronics & Information Technology, 2022, 44 (3) : 781-789.

[ 3 ] NGUYEN V, KHANH T T, VAN NAM P, et al. Towards flying mobile edge computing[ C]∥2020 International Conference on Information Networking ( ICOIN) . Piscataway: IEEE, 2020: 723-725.

[ 4 ] MA M L, GONG C Y, WU L T, et al. FLIRRAS: fast learning with integrated reward and reduced action space for online multitask offloading [ J ] . IEEE Internet of Things Journal, 2023, 10(6) : 5406-5417.

[ 5 ] AHANI G, YUAN D. BS-assisted task offloading for D2D networks with presence of user mobility[ C]∥2019 IEEE 89th Vehicular Technology Conference ( VTC2019-Spring) . Piscataway: IEEE, 2019: 1-5.

[ 6 ] LI N N, HAO W M, ZHOU F H, et al. Smart grid enabled computation offloading and resource allocation for SWIPT-based MEC system [ J] . IEEE Transactions on Circuits and Systems II: Express Briefs, 2022, 69(8) : 3610-3614.

[ 7 ] WANG L, HUANG P Q, WANG K Z, et al. RL-based user association and resource allocation for multi-UAV enabled MEC [ C ] ∥2019 15th International Wireless Communications & Mobile Computing Conference ( IWCMC) . Piscatawy: IEEE, 2019: 741-746.

[ 8 ] 王丙琛, 司怀伟, 谭国真. 基于深度强化学习的自动驾驶车控制算法研究[ J] . 郑州大学学报( 工学版) , 2020, 41(4) : 41-45, 80.

WANG B C, SI H W, TAN G Z. Research on autopilot control algorithm based on deep reinforcement learning [ J] . Journal of Zhengzhou University ( Engineering Science) , 2020, 41(4) : 41-45, 80.

[ 9 ] YE Y T, WEI W S, GENG D Q, et al. Dynamic coordination in UAV swarm assisted MEC via decentralized

deep reinforcement learning[ C]∥2020 International Conference on Wireless Communications and Signal Processing ( WCSP) . Piscataway: IEEE, 2020: 1064-1069.

[10] WANG L, WANG K Z, PAN C H, et al. Deep reinforcement learning based dynamic trajectory control for

UAV-assisted mobile edge computing[ J] . IEEE Transactions on Mobile Computing, 2022, 21(10) : 3536-3550.

[11] LILLICRAP T P, HUNT J J, PRITZEL A, et al. Continuous control with deep reinforcement learning[ EB / OL] . (2019- 07 - 05 ) [ 2024 - 06 - 01 ] . https:∥arxiv. org /abs / 1509. 02971.

[12] MAO S, HE S F, WU J S. Joint UAV position optimization and resource scheduling in space-air-ground integrated networks with mixed cloud-edge computing[ J] . IEEE Systems Journal, 2021, 15(3) : 3992-4002.

[13] LIU Q, SHI L, SUN L L, et al. Path planning for UAVmounted mobile edge computing with deep reinforcement learning[ J] . IEEE Transactions on Vehicular Technology, 2020, 69(5) : 5723-5728.

[14] ZHANG L, ZHANG Z Y, MIN L, et al. Task offloading and trajectory control for UAV-assisted mobile edge computing using deep reinforcement learning [ J] . IEEE Access, 2021, 9: 53708-53719.

[15] WANG Y T, CHEN W W, LUAN T H, et al. Task offloading for post-disaster rescue in unmanned aerial vehicles networks [ J] . IEEE / ACM Transactions on Networking, 2022, 30(4) : 1525-1539.

[16] 李斌, 刘文帅, 费泽松. 面向空天地异构网络的边缘计算部分任务卸载策略[ J] . 电子与信息学报, 2022, 44(9) : 3091-3098.

LI B, LIU W S, FEI Z S. Partial computation offloading for mobile edge computing in space-air-ground integrated

network[ J] . Journal of Electronics & Information Technology, 2022, 44(9) : 3091-3098.

[17] HUO Y, LIU Q Y, GAO Q H, et al. Joint task offloading and resource allocation for secure OFDMA-based mobile edge computing systems [ J] . Ad Hoc Networks, 2024, 153: 103342.

[18] XU Y C, CHEN L L, LU Z H, et al. An adaptive mechanism for dynamically collaborative computing power and task scheduling in edge environment [ J] . IEEE Internet of Things Journal, 2023, 10(4) : 3118-3129.

[19] 李斌. 基于多智能体强化学习的多无人机边缘计算任务卸载[ J] . 无线电工程, 2023, 53(12) : 2731-2740. LI B. Multi-agent reinforcement learning-based task offloading for multi-UAV edge computing [ J] . Radio Engi

neering, 2023, 53(12) : 2731-2740.

[20] LI W, ZHOU F, CHOWDHURY K R, et al. QTCP: adaptive congestion control with reinforcement learning

[ J] . IEEE Transactions on Network Science and Engineering, 2019, 6(3) : 445-458.

[21] WANG P, NI W L. An enhanced dueling double deep Qnetwork with convolutional block attention module for traffic signal optimization in deep reinforcement learning[ J] . IEEE Access, 2024, 12: 44224-44232.


相似文献/References:

[1]刘广瑞,王庆海,姚冬艳.基于改进人工蜂群算法的多无人机协同任务规划[J].郑州大学学报(工学版),2018,39(03):51.[doi:10.13705/j.issn.1671-6833.2017.06.026]
　Liu Guangrui,Wang Qinghai,Yao Dongyan.Multi-UAV Cooperative Mission Planning Based on Improved Artificial Bee Colony Algorithm[J].Journal of Zhengzhou University (Engineering Science),2018,39(pre1):51.[doi:10.13705/j.issn.1671-6833.2017.06.026]
[2]闫怡汝,王寅.基于鸽群优化的复杂环境下无人机侦查航迹优化 [J].郑州大学学报(工学版),2019,40(04):3.[doi:10.13705/j.issn.1671-6833.2019.04.016]
　Yan Yiru,Wang Yin. undefined Pigeon-inspired Optimization Based Trajectory Planning Methodfor UAVs in a Complex Urban Environment [J].Journal of Zhengzhou University (Engineering Science),2019,40(pre1):3.[doi:10.13705/j.issn.1671-6833.2019.04.016]
[3]余松森,龙嘉濠,周 诺,等.基于相机运动轨迹的鲁棒无人机航拍稳像算法[J].郑州大学学报(工学版),2024,45(05):77.[doi:10.13705/j.issn.1671-6833.2024.02.004]
　YU Songsen,LONG Jiahao,ZHOU Nuo,et al.Robust Algorithm for Aerial Video Stabilization of UAV Based onCamera Motion Trajectory[J].Journal of Zhengzhou University (Engineering Science),2024,45(pre1):77.[doi:10.13705/j.issn.1671-6833.2024.02.004]

更新日期/Last Update: 2024-11-12