We found a match
Your institution may have access to this item. Find your institution then sign in to continue.
- Title
Actor-critic objective penalty function method: an adaptive strategy for trajectory tracking in autonomous driving.
- Authors
Wang, Bo; Bai, Fusheng; Zhang, Ke
- Abstract
Trajectory tracking is a key technology for controlling the autonomous vehicles effectively and stably to track the reference trajectory. How to handle the various constraints in trajectory tracking is very challenging. The recently proposed generalized exterior point method (GEP) shows high computational efficiency and closed-loop performance in solving the constrained trajectory tracking problem. However, the neural networks used in the GEP may suffer from the ill-conditioning issue during model training, which result in a slow or even non-converging training convergence process and the control output of the policy network being suboptimal or even severely constraint-violating. To effectively deal with the large-scale nonlinear state-wise constraints and avoid the ill-conditioning issue, we propose a model-based reinforcement learning (RL) method called the actor-critic objective penalty function method (ACOPFM) for trajectory tracking in autonomous driving. We adopt an integrated decision and control (IDC)-based planning and control scheme to transform the trajectory tracking problem into MPC-based nonlinear programming problems and embed the objective penalty function method into an actor-critic solution framework. The nonlinear programming problem is transformed into an unconstrained optimization problem and employed as a loss function for model updating of the policy network, and the ill-conditioning issue is avoided by alternately performing gradient descent and adaptively adjusting the penalty parameter. The convergence of ACOPFM is proved. The simulation results demonstrate that the ACOPFM converges to the optimal control strategy fast and steadily, and perform well under the multi-lane test scenario.
- Subjects
REINFORCEMENT learning; NONLINEAR equations; NONLINEAR programming; AUTONOMOUS vehicles; ITERATIVE learning control
- Publication
Complex & Intelligent Systems, 2024, Vol 10, Issue 2, p1715
- ISSN
2199-4536
- Publication type
Article
- DOI
10.1007/s40747-023-01238-6