Application of multi-agent deep reinforcement learning method to solve the dynamic weapon target assignment problem
183 viewsDOI:
https://doi.org/10.54939/1859-1043.j.mst.94.2024.11-21Keywords:
Reinforcement Learning, command and control system, C4I, DWTA, DQL, OpenAI GymAbstract
This paper presents the Multi-Agent Deep Reinforcement Learning method to solve the dynamic weapon target assignment (DTWA) in the air defense command and control system. The weapon model is built based on predicting the optimal trajectory of air targets and the status of objects on the ground, as well as the optimal plan to coordinate the activities of weapons in the system. The weapon model is built on the OpenAI Gym library, describes the rules of the dynamic air defense combat environment and uses deep reinforcement learning algorithms (Deep Q-Leanring) to optimize the policy. Experimental simulation results with different air defense scenarios demonstrate that, after being trained, the deep reinforcement learning model of the air defense weapon has the ability to automatically analyze, perceive situations, and coordinate with other air defense weapons in the system, build a dynamic resistance interaction plan and select the optimal plan taking into account practical constraints so that the overall loss function has a minimum value for the entire combat process. Therefore, the reinforcement learning model has the ability to be applied to develop software modules to support decision-making in the air defense command and control system.
References
[1]. Truong, N.X., Phuong, P.K., Phuc, H.V., Tien, V.H., “Q-Learning Based Multiple Agent Reinforcement Learning Model for Air Target Threat Assessment,” in The International Conference on Intelligent Systems & Networks, (2023), https://doi.org/10.1007/978-981-99-4725-6_16. DOI: https://doi.org/10.1007/978-981-99-4725-6_16
[2]. Lloyd Hammond, “Application of a Dynamic Programming Algorithm for Weapon Target Assignment”, Edinburgh South Australia: Defence Science and Technology Group, (2016).
[3]. Mohammad Babul Hasan and Yaindrila Barua, “Weapon Target Assignment”, DOI: 10.5772/intechopen.93665, October 6th, (2020). DOI: https://doi.org/10.5772/intechopen.93665
[4]. Fredrik Johansson, Göran Falkman, “SWARD: System for weapon allocation research & development,” in Information Fusion (FUSION), DOI:10.1109/ICIF.2010.5712067. DOI: https://doi.org/10.1109/ICIF.2010.5712067
[5]. Yiping Lu, Danny Z. Chen, “A new exact algorithm for the Weapon-Target Assignment problem,” Elsevier Ltd, vol. Omega 98,102138, (2021), https://doi.org/10.1016/j.omega.2019.102138, 2019. DOI: https://doi.org/10.1016/j.omega.2019.102138
[6]. Yang Zhao, Yifei Chen, Ziyang Zhen and Ju Jiang, “Multi-weapon multi-target assignment based on hybrid genetic algorithm in uncertain environment,” International Journal of Advanced Robotic Systems, no. https://doi.org/10.1177/1729881420905922, (2020). DOI: https://doi.org/10.1177/1729881420905922
[7]. Elias Munapo, “Development of an accelerating hungarian method for assignment problems,” Eastern-European Journal of Enterprise Technologies, pp. 6-13, (2020). DOI: https://doi.org/10.15587/1729-4061.2020.209172
[8]. Yuan Zeng Cheng,.. “Weapon Target Assignment Problem Solving Based on Hungarian Algorithm,” Applied Mechanics and Materials, doi:10.4028/www.scientific.net/AMM.713-715.2041, (2015). DOI: https://doi.org/10.4028/www.scientific.net/AMM.713-715.2041
[9]. Hildegarde Mouton, Jan Roodt, Herman Le Roux, “Applying Reinforcement Learning to the Weapon Assignment Problem in Air Defence,” Journal of Military Studies, vol. 39 No. 2, (2011), DOI: https://doi.org/10.5787/39-2-115. DOI: https://doi.org/10.5787/39-2-115
[10]. Tong Wang, Liyue Fu, Zhengxian Wei, “Unmanned ground weapon target assignment based on deep Q-learning network with an improved multi-objective artificial bee colony algorithm,” Engineering Applications of Artificial Intelligence. https://doi.org/10.1016/j.engappai.2022.105612, (2023). DOI: https://doi.org/10.1016/j.engappai.2022.105612
[11]. Brian Gaudet, Kristofer Drozd, “Deep Reinforcement Learning for Weapons to Targets Assignment in a Hypersonic strike,” University of Arizona. doi:10.13140/RG.2.2.19047.62881, (2023).
[12]. Yuxi Li, “Deep Reinforcement Learning: An Overview,” https://arxiv.org/abs/1701.07274, (2018).
[13]. Greg Brockman, Vicki Cheung, Ludwig Pettersson, “OpenAI Gym,” https://arxiv.org/pdf/1606. 01540.pdf, (2016).
[14]. John Schulman, Filip Wolski, Prafulla Dhariwal, “Proximal Policy Optimization Algorithms,” OpenAI, no. https://arxiv.org/pdf/1707.06347.pdf, pp. 1-12, (2017).
