Seminar - Sequential Decision Making under Uncertainty: Optimality Guarantees, Compositional Learning, and Applications to Robotics and Ecology - Apr. 28
Michael H. Lim
PhD Candidate, EECS, University of California, Berkeley
Friday, April 28, 2023 | AERO 114 | 2:45 PM
Abstract: Sequential decision making under uncertainty mathematically captures the challenge of choosing actions at each step, while accounting for potential rewards and uncertainties an agent may encounter in the future. However, such sequential decision making problems with various sources of uncertainty are notoriously difficult to solve, especially when the state and observation spaces are continuous or hybrid, which is often the case for physical systems. Furthermore, modern problem settings require complex machinery to effectively handle complex data structures like image, text or audio inputs, while performing complicated reasoning such as localizing with noisy camera images or predicting intentions and locations of other agents. Modern approaches that involve artificial intelligence and machine learning methods provide powerful computational resources that can effectively manage the above challenges. Many of these decision making algorithms and machine learning technologies can either capture rigorous theoretical guarantees or empirical performance, but few capture both.
This talk addresses decision making under uncertainty from multiple angles: theoretical guarantees, integration with learning, and real world applications. It strikes a balance between mathematical analysis of the foundational partially observable Markov decision process (POMDP) framework, and enabling these techniques via integration with machine learning techniques through compositional learning. We first analyze novel POMDP solvers and their theoretical convergence properties and give a general result guaranteeing the accuracy of a particle belief POMDP approximation. We then introduce approaches for integrating model-based planning with learning-based components via compositional learning. Lastly, we propose a novel application area of sequential decision making in ecology, where we formulate the population coexistence control problem as an optimal path planning problem, and discuss the benefits and impact. Future work directions include theoretical developments, alternative approaches for compositional learning, and other avenues for impactful real world applications.
Bio: Michael H. Lim is a PhD student in Electrical Engineering and Computer Science at UC Berkeley with a focus in Control, Intelligent Systems, and Robotics (CIR). He is advised by Profs. Claire J. Tomlin and Zachary N. Sunberg and supported by the NSF Graduate Research Fellowship.