On Reverse-Ordered Implementation of Differential Dynamic Programming for Unconstrained Optimal Control Problem

Differential dynamic programming (DDP) has been widely employed to numerically solve finite-time nonlinear optimal control problem. The core of the DDP is to update state and input trajectories at each iteration by repeating two processes termed the backward pass (BP) and the forward pass (FP). While the conventional order of the passes, first-BP-second-FP, explicitly reveals a feedback control structure in the trajectory update, this ordering also introduces extra difficulty when the DDP is applied to constrained problems. In this paper, we study a reverse-ordered implementation of the DDP algorithm in a sense of the first-FP-second-BP, which brings a structural simplicity of the algorithm. It is seen in simulations for a simple example that this reverse-ordered DDP is able to find the optimal trajectory as the conventional DDP, and often experiences a larger transient or a dynamic exploration, while mathematical analysis for these phenomenons is left to future works.