A Sensitivity-Based View of Learning and Optimization

Xiren Cao
Department of Electrical and Electronic Engineering
Hong Kong University of Science and Technology


Wednesday, October 26, 2005
4:30 - 5:45 PM
Terman Engineering Center, Room 453


Abstract:

The subject of learning and optimization has been attracting wide attentions from researchers in many disciplines including control systems, operations research, and computer science. Areas such as perturbation analysis (PA), Markov decision processes (MDPs), reinforcement learning (RL), and adaptive control (AC), share a common goal: to optimize the system performance. In this talk, we present a sensitivity-based view of learning and optimization of stochastic dynamic systems.

1. We show that the sensitivity-based view provides a unified framework for learning and optimization, and many existing results in the area can be derived from the two types of performance sensitivity equations, one for performance derivatives and the other for performance differences.

2. We study the Markov decision problem by using the performance difference equations and show that the long-run average performance-, the bias-, and the nth-bias- optimization problems can be easily and intuitively solved by this approach with no discounting or Laurent expansion. This simplifies the proof of Veinott¢s early results, and we hope that it can help to popularize these important works.

3. We introduce the event-based learning and optimization approach, which may be applied to problems that cannot be solved by the standard MDPs.

---

Operations Research Colloquia: http://or.stanford.edu/oras_seminars.html