IEEE CIS > Technical Activities

Adaptive Dynamic Programming and Reinforcement Learning TC

(1) Officers

Derong Liu, Chair (2009) Department of Electrical and Computer Engineering (M/C 154) University of Illinois at Chicago 851 S. Morgan Street Chicago, IL 60607-7053, USA phone: (+1 312) 355 4475 fax: (+1 312) 996 6465 email: dliu .a_t. ieee.org www: liu.ece.uic.edu

Marco Wiering, Vice Chair Department of Information and Computing Sciences University Utrecht Padualaan 14, De Uithof Utrecht 3584CH, Netherlands phone: (+31) 30 2539209 fax: (+31) 30 2513791 email: marco .a_t. cs.uu.nl www: people.cs.uu.nl/marco/

Huaguang Zhang, Vice Chair College of Information Science and Engineering Northeastern University Shenyang, Liaoning 110004, China phone: (+86) 24 83687762 fax: (+86) 24 83671498 email: hgzhang .a_t. ieee.org www: www.eai.neu.edu.cn/zhanghuaguang1.asp

(2) Members

Charles W. Anderson USA email: anderson .a_t. cs.colostate.edu

S. N. Balakrishnan USA email: bala .a_t. mst.edu

Andrew G. Barto USA email: barto .a_t. cs.umass.edu

Hamid R. Berenji Intelligent Inference Systems 333 W. Maude Ave, Suite 105 Sunnyvale, CA 94085, USA phone: (+1 408) 730 8345 fax: (+1 408) 730 8550 email: berenji .a_t. sonic.net

Dimitri Bertsekas USA email: dimitrib .a_t. mit.edu

Xiren Cao Hong Kong email: eecao .a_t. ust.edu.hk

Daniela de Farias USA email: pucci .a_t. mit.edu

Thomas G. Dietterich USA email: tgd .a_t. cs.orst.edu

Hai-Bin Duan China email: hbduan .a_t. buaa.edu.cn

El-Sayed El-Alfy Saui Arbia email: alfy .a_t. kfupm.edu.sa

Silvia Ferrari Department of Mechanical Engineering & Materials Science Duke University Box 90300, Durham, NC 27708-0005 USA phone: 919-660-5484 fax: 919-660-8963 email: sferrari .a_t. duke.edu www: fred.mems.duke.edu/silviaferrari.html

Zeng-Guang Hou Institute of Automation The Chinese Academy of Sciences P.O. Box 2728 Beijing 100190, China phone: +86 (10) 6256 5502 fax: +86 (10) 6256 5502 email: zengguang.hou .a_t. ia.ac.cn www: compsys.ia.ac.cn/~hou/

Hossein Javaherian General Motors Research and Development USA email: hossein.javaherian .a_t. gm.com

George G. Lendaris Department of Electrical and Computer Engineering Portland State University P.O. Box 751 Portland, OR 97207, USA phone: (+1 503) 725 4988 email: lendaris .a_t. pdx.edu www: www.sysc.pdx.edu/faculty/Lendaris/lendaris.html

Frank Lewis USA email: lewis .a_t. uta.edu

Derong Liu Department of Electrical and Computer Engineering (M/C 154) University of Illinois at Chicago 851 S. Morgan Street Chicago, IL 60607-7053, USA phone: (+1 312) 355 4475 fax: (+1 312) 996 6465 email: dliu .a_t. ieee.org www: liu.ece.uic.edu

Evangelia Micheli-Tzanakou Department of Biomedical Engineering Rutgers University 617 Bowser Road Piscataway, NJ 08855-0909, USA phone: (+1 732) 445 2037 fax: (+1 732) 445 3753 email: etzanako .a_t. rci.rutgers.edu

Remi Munos France email: remi.munos .a_t. inria.fr

Hector D. Patino Argentina email: dpatino .a_t. inaut.unsj.edu.ar

Warren Powell USA email: powell .a_t. princeton.edu

Philippe Preux France email: philippe.preux .a_t. univ-lille3.fr

Danil Prokhorov Toyota Technical Center, Michigan USA email: dvprokhorov .a_t. gmail.com www: home.comcast.net/~dvp/

John Rust USA email: jrust .a_t. gemini.econ.umd.edu

Jagannathan Sarangapani University of Missouri-Rolla Rolla, MO, USA email: sarangap .a_t. mst.edu

Jennie Si Department of Electrical Engineering Arizona State University Tempe, AZ 85287, USA phone: (+1 480) 965 6133 fax: (+1 480) 965 2811 email: si .a_t. asu.edu www: www.fulton.asu.edu/~jenniesi/

Luis Enrique Sucar Mexico email: esucar .a_t. inaoep.mx

Csaba Szepesvari Cadana email: szepesva .a_t. cs.ualberta.ca

Emanuel Todorov USA email: todorov .a_t. cogsci.ucsd.edu

Benjamin Van Roy USA email: bvr .a_t. stanford.edu

Ganesh Kumar Venayagamoorthy University of Missouri-Rolla USA email: gkumar .a_t. ieee.org

Draguna Vrabie USA email: dvrabie .a_t. uta.edu

Paul Werbos National Science Foundation 4201 Wilson Boulevard, Room 675 Arlington, VA 22230, USA phone: (+1 703) 292 8339 fax: (+1 703) 292 9147 email: pwerbos .a_t. nsf.gov www: www.nsf.gov/staff/staff_bio.jsp?lan=pwerbos

Bernard Widrow Department of Electrical Engineering Stanford University 860 Lathrop Drive Stanford, CA 94305, USA phone: (+1 650) 857 9151 fax: (+1 650) 857 1783 email: widrow .a_t. stanford.edu www: www-isl.stanford.edu/~widrow

Marco Wiering Department of Information and Computing Sciences University Utrecht Padualaan 14, De Uithof Utrecht 3584CH, Netherlands phone: (+31) 30 2539209 fax: (+31) 30 2513791 email: marco .a_t. cs.uu.nl www: people.cs.uu.nl/marco/

Donald Wunsch Dept. of Electrical and Computer Engineering Missouri University of Science & Technology Rolla, USA email: dwunsch .a_t. mst.edu www: www.linkedin.com/in/wunsch

Gary Yen School of Electrical and Computer Engineering Oklahoma State University 202 Engineering South Stillwater, OK 74078-5032, USA phone: (+1 405) 744 7743 fax: (+1 405) 744 9198 email: gyen .a_t. okstate.edu www: www.okstate.edu/elec-engr/faculty/yen

Huaguang Zhang College of Information Science and Engineering Northeastern University Shenyang, Liaoning 110004, China phone: (+86) 24 83687762 fax: (+86) 24 83671498 email: hgzhang .a_t. ieee.org www: www.eai.neu.edu.cn/zhanghuaguang1.asp

(3) Task Forces

3.1 Complexity issues in ADP and RL

Remi Munos, Chair France email: remi.munos .a_t. inria.fr

Scope: Numerical complexity of solving ADP and RL problems: amount of numerical resources required to obtain near-optimal performances. Sample complexity for RL: number of calls to the generative model (or the real system) required to build a near-optimal approximations.

3.2 Important applications of ADP and RL

John Rust, Chair USA email: jrust .a_t. gemini.econ.umd.edu

Analysis of power grid cascades and other cascades in man-made systems, analysis of evolution of epidemics, evolution of the stock market, optimal trading of commodities. There are also a large numbers of applications in economics, management science and other areas, and often they are less abstract and theoretical and tries to make actual, measurable, concrete contributions to improve decision making by firms, governments and other organizations.

3.3 Representation in ADP and RL

Philippe Preux, Chair France email: philippe.preux .a_t. univ-lille3.fr

In ADP and RL, we need a function approximator to represent the learned function, either the value, or the policy, or a model of the dynamics. Tools used for such approximation includes neural networks and many others. There are also issues on how to represent a state in order to achieve the best learning curve. Many issues are intertwined here, ranging from fundamental issues, to algorithmic ones, and practical ones.