The Statistical Foundations of Learning to Control
Given the dramatic successes in machine learning and reinforcement learning (RL) over the past half decade, there has been a resurgence of interest in applying these techniques to continuous control problems in robotics, self-driving cars, and unmanned aerial vehicles. Though such applications appear to be straightforward generalizations of standard RL, few fundamental baselines have been established prescribing how well one must know a system in order to control it. In this talk, I will discuss the general paradigm for RL and how it is related to more classical concepts in control. I will then describe a contemporary view merging techniques from statistical learning theory and robust control to derive baselines for these continuous control problems. I will explore several examples that balance perception and action, and demonstrate finite sample tradeoffs between estimation and control performance. I will close by listing several exciting open problems that must be solved before we can build robust, safe learning systems that interact with an uncertain physical environment.