This paper deals with a finite-state, finiteaction discrete-time Markov decision model. A linear programming procedure is developed for the computation of optimal policies over the entire range of the discount factor. Furthermore, a procedure is presented for the computation of a Blackwell optimal policy.

OR Spektrum
Erasmus School of Economics

Hordijk, A, Dekker, R, & Kallenberg, L.C.M. (1985). Sensitivity analysis in discounted Markovian decison problems. OR Spektrum, 7(3), 143–151. doi:10.1007/BF01721353