TRACKING CONTROL FOR UNCERTAIN SURFACE VESSEL WITH EXTERNAL DISTURBANCE VIA ON-POLICY ADAPTIVE REINFORCEMENT LEARNING – RISE CONTROLLER
239 viewsKeywords:
Optimal Control; Adaptive Dynamic Programming (ADP); Surface Vessel (SV) systems; Robust Integral of the Sign of the Error (RISE); Trajectory Tracking Control.Abstract
This article addresses a trajectory tracking control approach for uncertain/disturbed surface vessels using the new structure of adaptive reinforcement learning (ARL) algorithm and Robust Integral of the Sign of the Error (RISE). To obtain an asymptotic tracking performance, a robust integral of the sign of the error (RISE) feedback term is introduced in feedback control design to compensate for the uncertain components in the model and external disturbances. The adaptive reinforcement learning technique is proposed by tuning simultaneously the actor-critic network to approximate the control policy and the cost function, respectively. The convergence of weight as well as tracking control problem was determined by theoretical analysis. Finally, the numerical example is investigated to validate the effectiveness of the proposed control scheme.
References
[1]. B. Xiao, X. Yang, and X. Huo, “A Novel Disturbance Estimation Scheme for Formation Control of Ocean Surface Vessels,” IEEE Transactions on Industrial Electronics, vol. 64, no. 6. (2017) pp. 4994–5003,
[2]. S. L. Dai, M. Wang, and C. Wang, “Neural Learning Control of Marine Surface Vessels with Guaranteed Transient Tracking Performance,” IEEE Transactions on Industrial Electronics, vol. 63, no. 3. (2016) pp. 1717–1727.
[3]. Z. Zhao, W. He, and S. S. Ge, “Adaptive neural network control of a fully actuated marine surface vessel with multiple output constraints,” IEEE Transactions on Control Systems Technology, vol. 22, no. 4. (2014) pp. 1536–1543.
[4]. C. S. W. He, Z. Yin, “Adaptive neural network control of a marine vessel with constraints using the asymmetric barrier Lyapunov function,” IEEE Trans. Cybern., vol. 47(7), (2017) pp. 1641–1651.
[5]. Z. Yin, W. He, C. Yang, and C. Sun, “Control Design of a Marine Vessel System Using Reinforcement Learning,” Neurocomputing, vol. 311 (2018) pp. 353–362.
[6]. Z. Yin, W. He, C. Sun, G. Li, and C. Yang, “Adaptive control of a marine vessel based on reinforcement learning,” Chinese Control Conference, CCC, vol. 2018-July. (2018) pp. 2735–2740.
[7]. G. Wen, S. S. Ge, C. L. P. Chen, F. Tu, and S. Wang, “Adaptive tracking control of surface vessel using optimized backstepping technique,” IEEE Trans. Cybern., vol. 49, no. 9, (2019) pp. 3420–3431.
[8]. R. C. Xinxin Guo, Weisheng Yan, “Integral Reinforcement Learning-Based Adaptive Systems With Unknown Control Directions,” IEEE Trans. Syst. Man, Cybern. Syst., vol. PP, (2019) pp. 1–10.
[9]. Z. Zheng, L. Ruan, M. Zhu, and X. Guo, “Reinforcement learning control for underactuated surface vessel with output error constraints and uncertainties,” Neurocomputing, vol. 399. (2020) pp. 479–490.
[10]. P. Walters, R. Kamalapurkar, F. Voight, E. M. Schwartz, and W. E. Dixon, “Online Approximate Optimal Station Keeping of a Marine Craft in the Presence of an Irrotational Current,” IEEE Trans. Robot., vol. 34, no. 2, (2018) pp. 486–496.
[11]. A. B. Martinsen, A. M. Lekkas, S. Gros, J. A. Glomsrud, and T. A. Pedersen, “Reinforcement Learning-Based Tracking Control of USVs in Varying Operational Conditions,” Frontiers in Robotics and AI, vol. 7 (2020).
[12]. S. Bhasin, R. Kamalapurkar, M. Johnson, K. G. Vamvoudakis, F. L. Lewis, and W. E. Dixon, “A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems,” Automatica, vol. 49, no. 1 (2013) pp. 82–92.
[13]. K. G. Vamvoudakis and F. L. Lewis, “Online actor critic algorithm to solve the continuous-time infinite horizon optimal control problem,” Proceedings of the International Joint Conference on Neural Networks (2009) pp. 3180–3187.
[14]. B. Xian, D. M. Dawson, M. S. De Queiroz, and J. Chen, “A Continuous Asymptotic Tracking Control Strategy for Uncertain Nonlinear Systems,” IEEE Trans. Automat. Contr., vol. 49, no. 7, (2004) pp. 1206–1211.
