Fan Lu

Fan Lu

Postdoc Researcher at UCSC

Talks and presentations

Accelerating Wound Healing using Deep Reinforcement Learning

November 18, 2023

Poster Presentation, Predictive Modeling in Biology and Medicine Conference, University of California, Riverside, CA

Description: We propose a deep reinforcement learning (DRL) based approach to expedite the wound healing process. Our method efficiently divides the vision-based control task into two distinct modules: a perception module and a controller module. By separating vision-based control into a perception module and a controller module, we train a DRL agent without sophisticated mathematical modeling. Our research demonstrates the algorithm’s convergence and establishes its robustness in accelerating the wound-healing process. The proposed DRL methodology showcases significant potential for expediting wound healing by effectively integrating perception, control, and predictive modeling, all while eliminating the need for intricate mathematical models.

Convex Q-Learning: Theory and Applications

October 30, 2023

Talk, AM Seminar at the University of California Santa Cruz, Santa Cruz, CA

Description: Finding optimal control policies is one of the most important tasks in reinforcement learning. It is well known that the extension of Watkins’ algorithm to general function approximation settings is challenging: does the projected Bellman equation have a solution? If so, is the solution useful in the sense of generating a good policy? And, if the preceding questions are answered in the affirmative, is the algorithm consistent? These questions are unanswered even in the special case of Q-function approximations that are linear in the parameter. The challenge seems paradoxical, given the long history of convex analytic approaches to dynamic programming.