Discount factor-based data-driven reinforcement learning cascade control structure for unmanned aerial vehicle systems

Abstract

This article investigates the discount factor-based data-driven reinforcement learning control (DDRLC) algorithm for completely uncertain unmanned aerial vehicle (UAV) quadrotors. The proposed cascade control structure of UAV is categorized with two control loops of attitude and position sub-systems, which are established the proposed discount factor-based DDRLC algorithm. Through the analysis of the Bellman function's time derivative from two perspectives, a revised Hamilton-Jacobi-Bellman (HJB) equation including a discount factor is developed. Then, in the view of off-policy consideration, an equation is formulated to simultaneously solve the approximate Bellman function and approximate optimal control law in the proposed DDRLC algorithm with guaranteed convergence. According to the modified state variables vector, the development of the discount factor-based DDRLC algorithm in each control loop is indirectly implemented by transforming the time-varying tracking error model into the time invariant system. Finally, a simulation study on the proposed discount factor-based DDRLC algorithm is provided to validate its effectiveness. To validate the tracking performance of the quadrotor, four performance indices are considered, including IAE_p=3.0527, IAE_Ω=0.1175, ITAE_p=1.8408, and ITAE_Ω=0.0144, where the subscript p denotes position tracking error and Ω denotes attitude tracking error.