Title: Worst-Case Control and Learning in Cyber-Physical Systems with Approximate Information States
Speaker: Dr. Aditya Dave, Postdoctoral Associate at the School for Civil and Environmental Engineering at Cornell University, Ithaca, USA.
Day, Date, and Time: Thursday 11th January, 2024, 09:30 am to 10:30 am
Venue: Seminar room IE 211, Second Floor, IEOR Building
Abstract: The advent of cyber-physical systems has revolutionized numerous applications, including connected and automated vehicles, networked control systems, medicine and healthcare, and the Internet of Things. These systems require new approaches that can utilize the improved computational capabilities of the cyber core to control their physical components, while accounting for partially observed states and uncertain disturbances in real-world implementation. This talk will present a theory for worst-case control and robust reinforcement learning in such partially observed systems. When the dynamics are known, a robust control strategy can be computed using a dynamic programming (DP) decomposition that uses the agent's entire memory. However, such an approach quickly becomes computationally infeasible. Instead, we will explore a more general framework for decision-making in such systems by developing the notions of information states and approximate information states. An information state is a compression of the memory that potentially yields a more computationally efficient DP decomposition. However, this does not sufficiently improve efficiency in complex systems and is difficult to utilize when the dynamics are unknown. Thus, we will introduce approximate information states, which can further improve computational efficiency with a bounded loss in worst-case performance of the approximate strategy. An important feature of the proposed framework is that approximate information states can be constructed using output data in control problems and can be learned from output data in reinforcement learning problems. The complete framework constitutes a principled approach to decision-making for robust control and learning in cyber-physical systems.
Bio: Dr. Aditya Dave is a Postdoctoral Associate at the School for Civil and Environmental Engineering at Cornell University, Ithaca, USA. He received his PhD in the Department of Mechanical Engineering at the University of Delaware, Newark, USA in 2023 and completed his B. Tech. in Mechanical Engineering from the Indian Institute of Technology Bombay, India, in 2016. Prior to his PhD degree, he worked with Cairn Oil and Gas, Gurugram, India from 2016-2017. His current research interests span several areas, including cyber physical systems, robust reinforcement learning, human-robot interaction, and equity in emerging mobility. He is a member of IEEE and serves as a reviewer in multiple IEEE journals and conferences.