---

AM 226: Introduction to Stochastic Control Theory 

• Instructor: Paul Dupuis, 37 Manning Street, Rm 204,  Tel: x33238, dupuis@cfm.brown.edu, Office hours: 1-2 Thursday and 11-12 Friday. 

• Homework: There will be 2-3 homework assignments.  These will be a major part of the course, and are mandatory.  

• Grading: Grading will be based on the homework and computation projects.

• Textbook:  Kushner and Dupuis, Numerical Methods for Stochastic Control Problems in Continuous Time, (2nd ed) Springer, 2001.

• Course outline:

1. Introduction: Applications and Elementary Examples
2. Theory of Controlled Markov Chains-Dynamic Programming Equations
3. Continuous Time Models-Controlled Diffusions
4. Standard Formulations of Stochastic Control
5. Connections with PDE
6. Approximation of Processes-Examples and General Methods
7. Dynamic Programming Equations for Markov Chains-Methods of Solution
8. Weak Convergence
9. Convergence Proofs
10. Special Topics (E.g., Reflected or Jump Processes)
    

Assignments



• Assignment 1
• Assignment 2
• Assignment 3
• Assignment 4



Solutions

• Assignment 1 Solutions

Computational Assignments

• Computational Assignment 1
• Computational Assignment 2
• Computational Assignment 3