Optimal and Robust Control

Software for calculus of variations

In the first part of this lecture we covered the very basics of calculus of variations as needed by some optimal control methods for continuous time systems. While the content of this introduction calls for picking just a pen and a sheet of paper, you might have some benefit from doing some tedious math (finding derivatives and integrals) using some computer algebra system such as Symbolic Math Toolbox for Matlab, Mathematica, Maple or Sympy (for Python).

I like using Mathematica (for which an institutional licence is available at CVUT) because it has some explicit support for variational methods: https://reference.wolfram.com/language/VariationalMethods/tutorial/VariationalMethods.html. But even without using this specialized functionality, evaluating the Euler-Lagrange equation is particularly easy because Mathematica has no problems to find a derivative a function with respect to a derivative of another function. Doing this in Symbolic Math Toolbox for Matlab, for example, is a pain.

But none of the above is actually formally required in this course. I am just giving you some tips for your further self-study.

Software for continuous-time LQR

In the second half of the lecture we used the key result of calculus of variations - Euler-Lagrange equation - to solve the continuous-time LQ-optimal control problem on a finite time interval. For both the fixed and free final time, the problem has been (re)formulated as a two-point boundary value problem. This is a special class of problems with differential equations for which the boundary conditions are determined at two different time instants - at the beginning and at the end of the interval. Hence, solving these is no longer possible with the classical ODE solvers such as ode45. Instead, some other solvers have to be used. In Matlab, dedicated solver(s) exist, see https://www.mathworks.com/help/matlab/math/boundary-value-problems.html.

Nontheless, for LTI systems and the popular quadratic costs, we showed that the linear two-point boundary value ODE can be massaged into an initial value ODE of second order - a differential Riccati equation. This one could be solved using the above mentioned ode45 or similar solver for initial value problems. Eventually, we studies the steady-state version of the problem, in other words, we extended the time horizon to infinity, in which case we could use an algebraic version of Riccati equation, for which a dedicated solver exists in Matlab - care (note that it has been recently deprecated in favor of icare). Note that there is indeed a difference between the discrete- and continous-time variants of algebraic Riccati equations! Should you aim just at solving the LQR problem, there is a wrapper for the care solver - lqr. You may also want to check the extension that allows to include an integral in the controller - lqi.