Professor: Dr. Thomas R. Ioerger
Office: 322C Bright Bldg.
Phone: 845-0161
email: ioerger@cs.tamu.edu
office hours: make appointment by email

Meeting: Tues/Thurs, 11:10-12:25, 126 Bright

Course Web Page: http://www.cs.tamu.edu/faculty/ioerger/cs631-spr11/index.html

Prerequisites: CPSC 625 (Artificial Intelligence) - or an equivalent AI course as an undergrad

Textbooks

The textbook used in the course (which will be liberally supplemented with papers from the literature) will be:

Wooldridge, M. (2002). An Introduction to Multiagent Systems. Wiley.

• Ferber, J. (1999). Multi-Agent Systems. Addison-Wesley.
• Weiss, G. (1999). Multi-Agent Systems. MIT Press.
• Bradshaw, G. (1997). Software Agents. AAAI Press.
• Huhns, M. and Singh, P. (1998). Reading in Agents. Morgan Kaufmann.
• Wooldridge, M. and Rao, A. (1999). Foundations of Rational Agency. Applied Logic Series, Vol. 14, Kluwer.
• Wooldridge, M. (2000). Reasoning about Rational Agents. MIT Press.

Grading

The course will be primarily project-based, and grades will be based on the quality of the projects implemented.

Goals of this Course

To learn about extended topics in AI, with a focus on intelligent agents. Specifically, we will cover the following major topics/perspectives:

1. Agents: characteristics, environments, and architectures
   • Reactive Agents
   • Knowledge-based agents
   • Planning and Agents
   • BDI Architecture (intentions and beliefs)
   • Agent Communication Languages and Protocols (e.g. KQML)
   • Utility-based Agents, Decision Theory (e.g. Markov Decision Processes)
2. Multi-Agent Systems and Cooperative Behavior
   • PGP - partial-global plans and plan interleaving
   • contract nets
   • DCSP - distributed constraint satisfaction algorithms
   • negotiation
   • auctions
   • voting
   • coalition formation
   • teamwork
   • swarms
3. Adaptive Agents - Reinforcement Learning

Schedule:

References

• Bell and Sehtares (2001). Avoiding Global Congestion Using Decentralized Adaptive Agents - related to El Farol Bar problem, adaptive agent algorithm
• Wolpert and Tumer (2001). Optimal Payoff Functions for Members of Collectives - related to El Farol Bar problem
• Hanson and Crutchfield (1997). Computational Mechanics of Cellular Automata: An Example.
• Agent-Oriented Programming (Yoav Shoham, 1993)
• The Model-Checker SPIN (Holzmann, 1997) - a nice overview with examples
• A Survey of Concurrent METATEM (Fisher, 1994) - a nice overview with examples
• Model Checking (Clarke and Schlingloff, 2001) - the ultimate review of model checking for temporal logics
• Bratman, Israel, and Pollock (1988)
• Ingrand et al (1992) - IEEE Expert, Ingrand et al (1996) - ICRA - a task-based agent system
• Rao and Georgeff (1991) - BDI model (beliefs/desires/intentions)
• Rao and Georgeff (1995)
• Cohen and Levesque (1990) - modal logic of intentions
• (Durfee and Lesser, 1991) - PGP (partial global planning)
• (Yokoo and Durfee, 1998) - DCSP (distributed constraint satisfaction)
• KQML, COOL - multi-agent communication language
• Smith, Reid G. 1980. The Contract Net Protocol: High-Level Communicaation and Control in a Distributed Problem Solver. IEEE Transactions on Computers, C-29(12), 1104--1113.
• E. Ephrati and J. S. Rosenschein. The clarke tax as a consensus mechanism among automated agents. In Proceedings of the National Conference on Artificial Intelligence, pages 173--178, San Jose, California, July 1991.
• Zlotkin, G., and Rosenschein, J. S., "Coalition, Cryptography, And Stability: Mechanisms For Coalition Formation In Task Oriented Domains", Proceedings of the Twelfth National Conference on Artificial Intelligence, pp. 432-437, 1994.
• O. Shehory and S. Kraus. Task allocation via coalition formation among autonomous agents. In Proceedings of IJCAI'95, pages 655--661, 1995.
• M. Tambe. Towards flexible teamwork. Journal of Artificial Intelligence Research, 7:83--124, 1997.
• Makoto Yokoo and Edmund H. Durfee. The distributed constraint satisfaction problem: Formalization and algorithms. IEEE Transactions on Knowledge and Data Engineering, 10(5), 1998.
• A. Rao and M. Georgeff, "Modeling rational agents within a BDI architecture," Proceedings of the Second International Conference on Principles of Knowledge Representation and Reasoning, Cambridge, MA, 1991, pp. 473-484.
• Durfee, E. H., and Lesser V. R. (1991). Partial Global Planning: A Coordination Framework for Distributed Hypothesis Formation, IEEE Trans. Syst. Man and Cyber. SMC-21(5) pp 1167-1183.
• E. H. Durfee and V.R. Lesser (1987). Using partial global plans to coordinate distributed problem solvers. In Proceedings of the Tenth International Joint Conference on Artificial Intelligence (IJCAI-87), pages 875-883.
• VSK logic
• Communicative Actions
• Sandholm, T. and Lesser, V. (2002). Leveled commitment contracting: A backtracking instrument for multiagent systems. AI Magazine, 23(3):89--100.
• Ephrati E., Rosenschein J.S. (1994). Divide and conquer in multiagent planning. In: Proceedings of the Twelfth National Conference on Artificial Intelligence (AAAI'94), pages 375-380.
• Gmytrasiewicz and Durfee (2001) (recursive modeling method, RMM)

Links

• JATLite - a java-based agent toolkit (download?)
• Jess

Elevator Links

• simple description
• Ashok Goel's assignment
• Crites and Barto (1996). Improving Elevator Performance Using Reinforcement Learning. NIPS, 8:1017-1023
• Crites and Barto (1998). Elevator Group Control Using Multiple Reinforcement Learning Agents. Machine Learning, 33:235-262
• elevator.tar - java code for simple simulator

MDP/RL Reading Material

• Barto and Sutton book
• Kaelbling, Littman, and Moore (1996). Reinforcement Learning: A Survey - simple descripition of Value and Policy iteration
• Szepesvari (2009). Algorithms for Reinforcement Learning - nice, well-rounded description of MDP and RL with up-to-date references
• Michael L. Littman, Thomas L. Dean, and Leslie Pack Kaelbling. On the complexity of solving Markov decision problems. (UAI-95) - abbreviated description of value and policy iteration

  ---