Maximum Principle and Dynamic Programming Viscosity Solution Approach : From Open-Loop to Closed-Loop

This book is concerned with optimal control problems of dynamical systems described by partial differential equations (PDEs). The content covers the theory and numerical algorithms, starting with open-loop control and ending with closed-loop control. It includes Pontryagin’s maximum principle and the Bellman dynamic programming principle based on the notion of viscosity solution. The Bellman dynamic programming method can produce the optimal control in feedback form, making it more appealing for online implementations and robustness. The determination of the optimal feedback control law is of fundamental importance in optimal control and can be argued as the Holy Grail of control theory.

The book is organized into five chapters. Chapter 1 presents necessary mathematical knowledge. Chapters 2 and 3 (Part 1) focus on the open-loop control while Chapter 4 and 5 (Part 2) focus on the closed-loop control. In this monograph, we incorporate the notion of viscosity solution of PDE with dynamic programming approach. The dynamic programming viscosity solution (DPVS) approach is then used to investigate optimal control problems. In each problem, the optimal feedback law is synthesized and numerically demonstrated. The last chapter presents multiple algorithms for the DPVS approach, including an upwind finite-difference scheme with the convergence proof. It is worth noting that the dynamic systems considered are primarily of technical or biologic origin, which is a highlight of the book.

This book is systematic and self-contained. It can serve the expert as a ready reference for control theory of infinite-dimensional systems. These chapters taken together would also make a one-semester course for graduate with first courses in PDE-constrained optimal control.

Bing Sun received his Ph.D. degree in operational research and control theory from the Academy of Mathematics and Systems Science, the Chinese Academy of Sciences, Beijing, China in 2005. From 2005 to 2006, he was a postdoctoral fellow at University of the Witwatersrand, South Africa. Since 2008, he has been with the Beijing Institute of Technology, first as an assistant professor (2008–2010), an associate professor (2010–2015) and subsequently a professor (since 2015) in mathematical system theory. He is the author or coauthor approximate 50 international peer/refereed journal papers. His research interests include the optimal control theory of systems described by partial differential equations and numerical solutions of optimal feedback control for both finite- and infinite dimensional systems. Dr. Sun received the Training Program Foundation for the Beijing Municipal Excellent Talents in 2012.

Bao-Zhu Guo received his Ph.D. in applied mathematics from the Chinese University of Hong Kong in 1991. From 1985 to 1987, he was a research assistant at Beijing Institute of Information and Control, China. During the period 1993–2000, he was with the Beijing Institute of Technology, first as an associate professor (1993–1998) and subsequently a professor (1998–2000). Since 2000, he has been with the Academy of Mathematics and Systems Science, the Chinese Academy of Sciences, where he is a research professor in mathematical system theory. He is the author or co-author of over 290 peer/refereed papers in international journals. His research interests include the control and application of infinite-dimensional systems. Dr. Guo received the One Hundred Talent Program from the Chinese Academy of Sciences (1999), and the National Science Fund for Distinguished Young Scholars (2003). In 2013, he was elected a member of the Academy of Science of South Africa (ASSAf).

Zhen-Zhen Tao received her Ph.D. degree in applied mathematics from the Beijing Institute of Technology, Beijing, China in 2021 and is now a lecturer in the School of Mathematics and Information Science, Henan Polytechnic University, Jiaozuo, China. Her research interests include numerical solutions of partial differential equations and optimal control of distributed parameter systems.