Integrated UAVs Communications in Cellular Network: Deployment and Optimization via Deep Reinforcement Learning Technique

Abstract

The thesis focuses on optimizing the coverage and capacity of wireless networks through the use of unmanned aerial vehicles (UAVs) in a cellular-connected en­ vironment. The limitations of UAVs in capturing large areas necessitate the deployment of multiple UAVs to support the system, with the UAV Base Station (UAV-BS) communicating with other UAVs. The positioning of UAVs is cru­ cial to maximize data communication rate and meet real-time requirements. To ensure quality of experience (QoE) in real-time video streaming, a coordination co-design between UAVs and the UAV-BS is implemented to capture dynamic firefighting areas, by optimal bit-rate and power allocation to ensure smoothness quality in multiple locations. The study also addresses the challenge of downlink interference to terrestrial users (TUEs) when BSs serve both TUEs and UAVs. An interference coordination mechanism is proposed to mitigate inter-cell inter­ ference and maximize radio connectivity for TUEs. Dynamic cell muting interfer­ ence and resource allocation scheduling schemes (MOSDS-DQN) are introduced, leading to a significant improvement in throughput and satisfactory level for both UAVs and TUEs. Conventional beam-sweeping approaches face challenges due to the high mobility and autonomous operation of UAVs. To address this, the deep reinforcement learning (DRL)-based framework using hierarchical Deep Q- Network (hDQN) is proposed for UAV-BS beam alignment in a rnmWave radio setting. The framework utilizes location information to maximize beamforming gain during communication requests. To improve convergence time, the convo­lution neural network radio mapping and hDQN-based framework (hDRM) are employed. Simulation results showed that QoE is improved 12% compared to the non-learning algorithm with 41% improvement of the long-term video smooth­ ness. The proposed MOSDS-DQN showed 18% improvement compared to the DQN algorithm. The proposed hDRM framework improved 63% over the con­ verging time compared to vanilla hDQN approaches under real-time conditions.

Overall, the thesis contributes to the optimal positioning of UAVs and BSs, dy­namic bit-rate selection, interference mitigation, and efficient beam alignment using advanced techniques such as coordination co-design, dynamic scheduling, and deep reinforcement learning. These approaches enhance the performance and coverage of UAV-UEs, mitigate interference, and improve the overall efficiency of wireless networks in dynamic environments.

Date of Award	1 Sept 2023
Original language	English
Awarding Institution	King's College London
Supervisor	Hak-Keung Lam (Supervisor) & Yansha Deng (Supervisor)