A robust control algorithm for single input single output dynamic object based on table-based Q-method of reinforcement learning

The article provides an overview in the field of dynamic object control systems based on reinforcement learning. Based on the analysis, it is concluded that the development of control methods based on reinforcement learning is relevant. The article proposes an intelligent algorithm for robust control of stable dynamic objects with one input and one output, based on the tabular Q-learning method of zero order. The algorithm stabilizes the output value of the control object with a given error if the parameters and external disturbances of the object are piecewise constant unknown quantities, and the state vector is measurable. The novelty of the proposed algorithm lies in a new incremental method of control formation, which allows, based on a set of three possible actions, to stabilize the control object. The proposed method of forming a set of control actions makes it possible to ensure the required accuracy of stabilizing the output of an object by changing the amplitude of the control increment. The proposed algorithm has high computational efficiency. After training, the control calculation is reduced to calculating indexes based on measurement results, reading data from memory based on calculated indexes, and finding the maximum value in a small vector. For a discrete description of the control object, the conditions of convergence of the learning algorithm and the limitation of the control error are investigated. The developed algorithm is demonstrated by the example of the synthesis of robust control of a DC motor with independent excitation. In the course of numerical simulation, the quality of a closed system is investigated when the parameters and the control action change. The analysis of the simulation results allows us to draw conclusions about the effectiveness of the synthesized algorithm. The article also provides the results of a real experiment that demonstrate the technical feasibility of the algorithm obtained. This issue is important, since the analysis of sources shows an almost complete lack of technical implementation of control systems for dynamic objects synthesized using reinforcement learning methods.