IEOR 268: Applied Dynamic Programming

Instructor: Javad Lavaei
Time: Tuesdays and Thursdays, 9:30-11am
Location: 3109 Etcheverry
TA: Salar Fattahi (fattahi AT
Instructor's Office Hours: Mondays, 11am-noon (4121 Etcheverry)
TA's Office Hours: Fridays, 10-11am (4176-A Etcheverry)
Grading Policy:

  • 10% active participation

  • 40% homework

  • 50% project


This course provides a fundamental understanding of dynamic programming and its applications in different areas. Some of the topics covered in this course are as follows:

  • Discrete-time deterministic and stochastic problems

  • Continuous-time deterministic and stochastic problems

  • Deterministic and stochastic shortest path problems

  • Finite-horizon problems

  • Infinite-horizon discounted and undiscounted problems

  • Problems with perfect state information

  • Problems with imperfect state information

  • Approximate dynamic programming


  • “Dynamic Programming and Optimal Control” (2 volume set) by Dimitri P. Bertsekas, Athena Scientific, 4th Edition.