Papers by Nahid Parvaresh
IEEE Open Journal of the Communications Society
Uncrewed aerial vehicle-mounted base stations (UAV-BSs), also know as drone base stations, are co... more Uncrewed aerial vehicle-mounted base stations (UAV-BSs), also know as drone base stations, are considered to have promising potential to tackle the limitations of ground base stations. They can provide cost-effective Internet connection to users that are out of infrastructure. They can also take over quickly as service providers when ground base stations fail in an unanticipated manner. UAV-BSs benefit from their mobile nature that enables them to change their 3D locations if the demand profile changes rapidly. In order to effectively leverage the mobility of UAV-BSs so as to maximize the performance of the network, 3D location of UAV-BSs requires continuous optimization. However, solving the optimization problem of UAV-BSs is NP-hard with no deterministic solution in polynomial time. In this paper, we propose a continuous actor-critic deep reinforcement learning solution in order to solve the location optimization problem of UAV-BSs in the presence of mobile endpoints. The simulation results show that the proposed model significantly improves the network performance compared to Q-learning, deep Q-learning and conventional algorithms. While the Q-learning and deep Q-learning-based baselines reach the sum data rate of 35 Mbps and 42 Mbps respectively, our proposed ACDQL-based strategy maximizes the sum data rate of endpoints to 45 Mbps. Furthermore, the proposed ACDQL-based methodology reduces the convergence time of the UAV-BS placement optimization by 85 percent compared to the Q-learning and deep Q-learning baselines.
Proceedings of the 20th ACM International Symposium on Mobility Management and Wireless Access on 20th ACM International Symposium on Mobility Management and Wireless Access
Uncrewed aerial vehicle-mounted base stations (UAV-BS) have recently attracted significant attent... more Uncrewed aerial vehicle-mounted base stations (UAV-BS) have recently attracted significant attention in order to assist ground base stations (BSs) and provide Internet access to users. UAV-BSs benefit from their mobility nature in the air and are able to constantly move towards the locations where the demand is higher. However, finding the optimal location of UAV-BSs and maintaining it is an NP-hard problem that has no deterministic solution in polynomial time. In this paper, we exploit reinforcement learning (RL) in order to solve the optimization problem of UAV-BSs and find their optimal location in the presence of mobile User Equipment (UEs). We consider UAV-BS as the agent of RL and deploy two algorithms, i.e. Q-learning and deep Q-learning in order to solve the location optimization problem of UAV-BSs. Through simulations, we show that the proposed DQL model with a continuous state space including the mobility information of users can effectively adapt to the environmental changes and improve the user data rate by 46%, packet loss ratio by 70%, and transmission delay by 60%. CCS CONCEPTS • Human-centered computing → Ubiquitous and mobile computing; • Networks → Location based services; • Computing methodologies → Cluster analysis.
Uploads
Papers by Nahid Parvaresh