Title: Towards AI Incorporated Optimization Techniques - Integrating Bandits into Gradient Descent and Random Optimization
Speaker: Syed Shahul Hameed
Venue: Seminar hall, second floor, IEOR Building
Day, Date, Time: Friday, 9th August 2024, 11:00AM-12:00PM
Abstract: The essence of Artificial Intelligence (AI) thrives on optimization. The `Intelligence' in AI is practically realized through diverse optimization algorithms. What if the realm of optimization is endowed with the same intelligence? By intelligence, we specifically refer to the capability to `Learn.' Learning, as in machine learning, when incorporated into optimization algorithms, opens a wide avenue of possibilities to self-configure and adapt its performance as the search process evolves.
In this work, we identify the suitability of Reinforcement Learning (RL) as a technique that can be conveniently integrated into different optimization algorithms at a sub-algorithmic level. We specifically leverage the Multi-Armed Bandit framework to incorporate the `capability to learn' into the optimization algorithms, allowing the latter to self-configure its algorithmic parameters. Elementary optimization algorithms, like gradient descent and random search methods, are studied in this context.
The fundamental goal of my PhD work is to introduce bandit techniques into optimization algorithms in a seamless manner, without adding additional complexity. Efforts are made to ensure that, this integration is minimally invasive as possible, so that the elegance of the original algorithm is preserved. This notion of using bandits for hyperparameter tuning can possibly be extended to the ML setting, where the tuning process is quite involved. A simple standalone bandit may not suffice for this purpose. Multiple MABs can be strategically employed to conduct the tuning process in machine learning, leading to the possibility of studying and designing intricate mechanisms to coordinate the bandits.
Brief Bio: Syed Shahul Hameed A S (CS20D1003) is a full-time Ph.D. research scholar in the Computer Science and Engineering Department at the National Institute of Technology Puducherry, Karaikal - 609609. He has completed his Bachelor of Technology in Computer Science and Engineering from Perunthalaivar Kamarajar Institute of Engineering and Technology (PKIET), Nedugadu, Karaikal. He did his Master of Technology in Computer Science and Engineering from the Indian Institute of Technology Tirupati (IIT Tirupati), Andhra Pradesh, India. His postgraduate dissertation was on applying reinforcement learning techniques for throughput optimization in coexisting WiFI and LTE-U. His research interests include function optimization, reinforcement learning, and bandit techniques. During his doctoral research at NIT Puducherry, he has published four papers in reputed peer-reviewed journals.