IEEE HomeSearch IEEEShop IEEEIEEE Web AccountContact IEEE
IEEE Computational Intelligence Society
     f=0127

IEEE CIS > Technical Activities

Adaptive Dynamic Programming and Reinforcement Learning TC

(1) Officers

Marco WieringMarco Wiering, Chair (2010)
Department of Artificial Intelligence
University of Groningen
Padualaan 14, De Uithof
Groningen, Nijenborgh 9 9700AK, Netherlands
phone: +31(0)50-3636956
fax: +31(0)50-3636687
email: m.a.wiering .a_t. rug.nl
www: www.ai.rug.nl/~mwiering
Huaguang ZhangHuaguang Zhang, Vice Chair
Department of Electrical Engineering
Northeastern University
Wenhua Road 3-11#, Heping District
Shenyang, Liaoning 110819, China
phone: +86-24-83687762
fax: +86-24-83689605
email: hgzhang .a_t. ieee.org
www: www.eai.neu.edu.cn/zhanghuaguang1.asp
Damien ErnstDamien Ernst, Vice Chair
Systems and Modeling Research Unit
University of Liege
Institut Montefiore
Liege B-4000, Belgium
phone: +32 4 366 9518
fax: +32 4 366 2984
email: dernst .a_t. ulg.ac.be
www: www.montefiore.ulg.ac.be/~ernst/

(2) Members

Charles W. AndersonCharles W. Anderson
USA
email: anderson .a_t. cs.colostate.edu
Robert BabuskaRobert Babuska
Delft Center for Systems and Control
Delft University of Technology
Mekelweg 2, 2628 CD Delft
Delft 2600 GA, The Netherlands
phone: +31 15 2785117
fax: +31 15 2786679
email: r.babuska .a_t. tudelft.nl
www: www.dcsc.tudelft.nl/~babuska/
Andrew G. BartoAndrew G. Barto
University of Massachusetts at Amherst
USA
email: barto .a_t. cs.umass.edu
Anna BazzanAnna Bazzan
Instituto de Informatica
Porto Alegre, RS
Brazil
email: bazzan .a_t. inf.ufrgs.br
www: www.inf.ufrgs.br/~bazzan/
Dimitri BertsekasDimitri Bertsekas
USA
email: dimitrib .a_t. mit.edu
Lucian BusoniuLucian Busoniu
Delft Center for Systems and Control
Delft University of Technology
Mekelweg 2
Delft 2628 CD, The Netherlands
phone: +31 15 2788573
fax: +31 15 2786679
email: i.l.busoniu .a_t. tudelft.nl
www: www.dcsc.tudelft.nl/~lbusoniu/home.php
Daniela de FariasDaniela de Farias
USA
email: pucci .a_t. mit.edu
El-Sayed M. El-AlfyEl-Sayed M. El-Alfy
College of Computer Sciences and Engineering
King Fahd University of Petroleum and Minerals
P.O. Box 371
Dhahran 31261, Saui Arbia
phone: +9663-860-1930
fax: +9663-860-2174
email: alfy .a_t. kfupm.edu.sa
www: faculty.kfupm.edu.sa/ics/alfy
Damien ErnstDamien Ernst
Systems and Modeling Research Unit
University of Liege
Institut Montefiore
Liege B-4000, Belgium
phone: +32 4 366 9518
fax: +32 4 366 2984
email: dernst .a_t. ulg.ac.be
www: www.montefiore.ulg.ac.be/~ernst/
Silvia FerrariSilvia Ferrari
Department of Mechanical Engineering & Materials Science
Duke University
Box 90300, Durham, NC 27708-0005
USA
phone: 919-660-5484
fax: 919-660-8963
email: sferrari .a_t. duke.edu
www: fred.mems.duke.edu/silviaferrari.html
Zeng-Guang HouZeng-Guang Hou
Institute of Automation
The Chinese Academy of Sciences
P.O. Box 2728
Beijing 100190, China
phone: +86 (10) 6256 5502
fax: +86 (10) 6256 5502
email: zengguang.hou .a_t. ia.ac.cn
www: compsys.ia.ac.cn/~hou/
George G. LendarisGeorge G. Lendaris
Department of Electrical and Computer Engineering
Portland State University
P.O. Box 751
Portland, OR 97207, USA
phone: (+1 503) 725 4988
email: lendaris .a_t. sysc.pdx.edu
www: www.sysc.pdx.edu/faculty/Lendaris/lendaris.html
Frank LewisFrank Lewis
Department of Electrical Engineering
University of Texas at Arlington
416 Yates Street
Arlington, TX 76011, USA
phone: 817-272-5972
fax: 817-272-5989
email: lewis .a_t. uta.edu
www: arri.uta.edu/acs/bios/lewis.htm
Derong LiuDerong Liu
Institute of Automation
Chinese Academy of Sciences
Beijing 100190, China
Department of Electrical and Computer Engineering
University of Illinois
Chicago, IL 60607, USA
email: ieeetnn .a_t. gmail.com
www: www.ece.uic.edu/~derong/
Haibo HeHaibo He
Electrical and Computer Engineering
Stevens Institute of Technology
Burchard 412
Hoboken, NJ 07030, USA
phone: (201)216-8057
fax: (201)216-8246
email: hhe .a_t. stevens.edu
www: www.ece.stevens-tech.edu/~hhe/
Marcus HutterMarcus Hutter
Research School of Information Sciences and Engineering
Australian National University
Corner of North and Daley Road
Canberra ACT 0200, Australia
phone: +61(0)2 612 51605
fax: +61(0)2 612 58651
email: marcus.hutter .a_t. anu.edu.au
www: www.hutter1.net
Remi MunosRemi Munos
France
email: remi.munos .a_t. inria.fr
Hector D. PatinoHector D. Patino
Argentina
email: dpatino .a_t. inaut.unsj.edu.ar
Jan PetersJan Peters
Department for Empirical Inference and Machine Learning
Max-Planck-Institute for Biological Cybernetics
Spemannstrasse 38
Tubingen 72076, Germany
phone: +49 7071 601585
fax: +49 7071 601552
email: Jan.peters .a_t. tuebingen.mpg.de
www: www.kyb.mpg.de/~jpeters
Warren PowellWarren Powell
USA
email: powell .a_t. princeton.edu
Philippe PreuxPhilippe Preux
France
email: philippe.preux .a_t. univ-lille3.fr
Danil ProkhorovDanil Prokhorov
Toyota Technical Center, Michigan
USA
email: dvprokhorov .a_t. gmail.com
www: home.comcast.net/~dvp/
John RustJohn Rust
USA
email: jrust .a_t. gemini.econ.umd.edu
Jagannathan SarangapaniJagannathan Sarangapani
Electrical and Computer Engineering
Missouri university of Science and Technology
1870 Miner Circle
Rolla, MO 65409, USA
phone: (573)341-6775
fax: (573)341-4532
email: sarangap .a_t. mst.edu
www: web.mst.edu/~sarangap/
Stefan SchaalStefan Schaal
Computer Science, Neuroscience , and Biomedical Engineering
University of Southern California
3710 S. McClintock Ave
Los Angeles, California 90089-2905, USA
phone: 310 740 1976
fax: 213 740 1510
email: sschaal .a_t. usc.edu
www: www-clmc.usc.edu/~sschaal/
Jennie SiJennie Si
Department of Electrical Engineering
Arizona State University
Tempe, AZ 85287, USA
phone: (+1 480) 965 6133
fax: (+1 480) 965 2811
email: si .a_t. asu.edu
www: www.fulton.asu.edu/~jenniesi/
Luis Enrique SucarLuis Enrique Sucar
Mexico
email: esucar .a_t. inaoep.mx
Csaba SzepesvariCsaba Szepesvari
Cadana
email: szepesva .a_t. cs.ualberta.ca
Emanuel TodorovEmanuel Todorov
USA
email: todorov .a_t. cogsci.ucsd.edu
Benjamin Van RoyBenjamin Van Roy
USA
email: bvr .a_t. stanford.edu
Athanasios V. VasilakosAthanasios V. Vasilakos
Department of Computer and Telecommunications Eng
University of Western Macedonia,Greece
Krinis 3
N.Erythraia, Greece 14671, Greece
phone: +30 6977449705
email: vasilako .a_t. ath.forthnet.gr
Ganesh Kumar VenayagamoorthyGanesh Kumar Venayagamoorthy
Real-Time Power and Intelligent Systems Laboratory
Missouri University of Science and Technology
Rolla
Missouri, USA
email: gkumar .a_t. ieee.org
Draguna VrabieDraguna Vrabie
Automation and Robotics Research Institute
University of Texas at Arlington
7300 Jack Newell Blvd. S.
Fort Worth, TX 76118, USA
phone: 817-272-5971
fax: 817-272-5938
email: dvrabie .a_t. uta.edu
www: www.uta.edu/ra/real/editprofile.php?pid=2889
Paul WerbosPaul Werbos
National Science Foundation
4201 Wilson Boulevard, Room 675
Arlington, VA 22230, USA
phone: (+1 703) 292 8339
fax: (+1 703) 292 9147
email: pwerbos .a_t. nsf.gov
www: www.nsf.gov/staff/staff_bio.jsp?lan=pwerbos
Shimon WhitesonShimon Whiteson
Department of Computer Science
University of Amsterdam
Science Park 107
Amsterdam 1098 XG, Netherlands
phone: +31 (0)20.525.8701
fax: +31 (0)20.525.7490
email: s.a.whiteson .a_t. uva.nl
www: staff.science.uva.nl/~whiteson/Shimon_Whiteson/Home.html
Bernard WidrowBernard Widrow
Department of Electrical Engineering
Stanford University
860 Lathrop Drive
Stanford, CA 94305, USA
phone: (+1 650) 857 9151
fax: (+1 650) 857 1783
email: widrow .a_t. stanford.edu
www: www-isl.stanford.edu/~widrow
Marco WieringMarco Wiering
Department of Artificial Intelligence
University of Groningen
Padualaan 14, De Uithof
Groningen, Nijenborgh 9 9700AK, Netherlands
phone: +31(0)50-3636956
fax: +31(0)50-3636687
email: m.a.wiering .a_t. rug.nl
www: www.ai.rug.nl/~mwiering
Donald C. Wunsch IIDonald C. Wunsch II
Dept. of Electrical and Computer Engineering
Missouri University of Science & Technology
301 W. 16th St, 131 EECH
Rolla MO 65409, USA
phone: 573-341-4521
fax: 573-341-4532
email: wunsch .a_t. ieee.org
www: www.linkedin.com/in/wunsch
Xin XuXin Xu
Institute of Automation
National University of Defense Technology
Changsha 410073, China
email: xinxu .a_t. nudt.edu.cn
www: jilsa.net/xinxu.html/
Huaguang ZhangHuaguang Zhang
Department of Electrical Engineering
Northeastern University
Wenhua Road 3-11#, Heping District
Shenyang, Liaoning 110819, China
phone: +86-24-83687762
fax: +86-24-83689605
email: hgzhang .a_t. ieee.org
www: www.eai.neu.edu.cn/zhanghuaguang1.asp

(3) Task Forces

3.1 Important applications of ADP and RL

John RustJohn Rust, Chair
USA
email: jrust .a_t. gemini.econ.umd.edu

Analysis of power grid cascades and other cascades in man-made systems, analysis of evolution of epidemics, evolution of the stock market, optimal trading of commodities. There are also a large numbers of applications in economics, management science and other areas, and often they are less abstract and theoretical and tries to make actual, measurable, concrete contributions to improve decision making by firms, governments and other organizations.


3.2 Reinforcement Learning and Function Approximation

Robert BabuskaRobert Babuska, Chair
Delft Center for Systems and Control
Delft University of Technology
Mekelweg 2, 2628 CD Delft
Delft 2600 GA, The Netherlands
phone: +31 15 2785117
fax: +31 15 2786679
email: r.babuska .a_t. tudelft.nl
www: www.dcsc.tudelft.nl/~babuska/
Damien ErnstDamien Ernst, Vice Chair
Systems and Modeling Research Unit
University of Liege
Institut Montefiore
Liege B-4000, Belgium
phone: +32 4 366 9518
fax: +32 4 366 2984
email: dernst .a_t. ulg.ac.be
www: www.montefiore.ulg.ac.be/~ernst/

In ADP and RL, we need a function approximator to represent the learned function, either the value, or the policy, or a model of the dynamics. Tools used for such approximation includes neural networks and many others. There are also issues on how to represent a state in order to achieve the best learning curve. Many issues are intertwined here, ranging from fundamental issues, to algorithmic ones, and practical ones.


3.3 Robot Reinforcement Learning

Jan PetersJan Peters, Chair
Department for Empirical Inference and Machine Learning
Max-Planck-Institute for Biological Cybernetics
Spemannstrasse 38
Tubingen 72076, Germany
phone: +49 7071 601585
fax: +49 7071 601552
email: Jan.peters .a_t. tuebingen.mpg.de
www: www.kyb.mpg.de/~jpeters
Stefan SchaalStefan Schaal, Vice Chair
Computer Science, Neuroscience , and Biomedical Engineering
University of Southern California
3710 S. McClintock Ave
Los Angeles, California 90089-2905, USA
phone: 310 740 1976
fax: 213 740 1510
email: sschaal .a_t. usc.edu
www: www-clmc.usc.edu/~sschaal/

Efficient self-improvement by trial and error is a key ability to allow robots to adapt to their environment and to learn new tricks. Reinforcement learning offers some of the most general tools in order to fomulate such robot learning problems while robotics is in theory a natural application domain for reinforcement learning. However, most reinforcement learning methods cannot be applied straightforwardly in robotics as the real-world constraints of the domain are creating exceedingly complex scenarios. On the other hand, robotics offers an enormous source of inspirations to reinforcement learning, Hence, it is essential to bring the insights from robotics into reinforcement learning and to create domain-appropriate reinforcement learning methods for robotics.