IEOR 268: Applied Dynamic Programming

Instructor: Javad Lavaei
Time: Tuesdays and Thursdays, 9:30-11 am
Location: 3113 Etcheverry
Office Hours: Mondays, 11am-noon
TA: Sheng Liu (lius10 AT
Grading Policy:

  • 10% active participation

  • 25% homework

  • 65% term project


This course provides a fundamental understanding of dynamic programming and its applications in different areas. Some of the topics covered in this course are as follows:

  • Discrete-time deterministic and stochastic problems

  • Continuous-time deterministic and stochastic problems

  • Deterministic and stochastic shortest path problems

  • Finite-horizon problems

  • Infinite-horizon discounted and undiscounted problems

  • Problems with perfect state information

  • Problems with imperfect state information

  • Approximate dynamic programming


  • “Dynamic Programming and Optimal Control” (2 volume set) by Dimitri P. Bertsekas, Athena Scientific, 4th Edition.

  • “Approximate Dynamic Programming: Solving the Curses of Dimensionality” by Warren B. Powell, 2nd Edition (optional).