Optimal and Robust Control

Assigned (compulsory) reading

Read the lecture notes [1] authored by the lecturer of this course.

Recommended (not compulsory) further reading

The content of this lecture is again very standard and is discussed by a number of books and online resources. However, in preparation of this lecture we took much inspiration in Liberzon's book [2], in particular chapters 3 (application of calculus of variations to general problem of optimal control) and chapter 4 (Pontryagin's principle). We did not talk about the proof of Pontryagin's principle at the lecture and we do not even command the students to go through the proof in the book. Understanding the result, its roots in calculus of variations and how it removes the deficiencies of the calculus of variations based results will suffice for our purposes.

Other treatments of this topic are offered by the other recommendable (and affordable) book [3] on optimal control.

The transition from the calculus of variations to the optimal control, especially when it comes to the definition of Hamiltonian, is somewhat tricky. Unfortunately, it is not discussed satisfactorily in the literature. Even Liberzon leaves it as an (unsolved) exercise (3.5 and 3.6) to the student. Other major textbooks avoid the topic altogether. The only treatment can be found in the famouse journal paper [4] by Sussmann and Willems, in particular in the section ``The first fork in the road: Hamilton'' on page 39. The issue is so delicate that they even propose to distinguish the two types of Hamiltonian by referring to one as control Hamiltonian. A short discussion of this topic also appeared in Wikipedia [5].

The time-optimal control for linear systems, in particular bang-bang control for a double integrator is described in section 4.4.1 and 4.4.2 in Liberzon. The material is quite standard and can be found in many books and lecture notes. What is not covered, however, is the fact that without any adjustment, the bang bang control is very troublesome from an implementation viewpoint. A dedicated research thread has evolved, especially driven by the needs of hard disk drive industry, which is called (a)proximate time-optimal control (PTOS). Many dozens of papers can be found with this keyword in the title.

Among the numerous online resources, the lecture notes by Lawrence C. Evans from UC Berkeley [6] can be recommended.

1. Z. Hurák. Pontryagin's principle of maximum; time-optimal control. Lecture notes Optimal and Robust Control. Czech Technical University in Prague, 2017. Available [ONLINE] at  moodle.fel.cvut.cz/courses/b3m35orr.
2. D. Liberzon. Calculus of Variations and Optimal Control Theory - A Concise Introduction. Princeton University Press, 2012. Preprint available freely [ONLINE] at http://liberzon.csl.illinois.edu/teaching/cvoc/cvoc.html.
3. D. E. Kirk. Optimal Control Theory - An Introduction. Dover, 2004
4. H. J. Sussmann, J. C. Willems.  300 years of optimal control: from the brachystochrone to the maximum principle. IEEE Control Systems, Volume: 17, Issue: 3, Jun 1997. DOI: 10.1109/37.588098.
5. Wikipedia. https://en.wikipedia.org/wiki/Hamiltonian_%28control_theory%29.
6. L. Evans. Introduction to Mathematical Optimal Control Theory. Version 0.2, available [ONLINE] at https://math.berkeley.edu/~evans/control.course.pdf.