Stochastic dynamic programming

The words ''stochastic dynamic programming'' were invented by the American mathematician Richard Bellman, who was both incredibly prolific and incredibly creative (often these do not go together).

Bellman invented the name somewhat before he had the method (Bellman 1984) and this makes sense if you think that the method is based on two principles: (1) with probability 1 something will happen; and (2) whatever happens, act optimally. The origin of the method goes back at least to the 1800s and the Irish mathematician Hamilton in his study of the motion of the planets (Courant and Hilbert 1962). The method of stochastic dynamic programming as a tool in behavioral ecology was popularized by Colin Clark, Alasdair Houston, John McNamara and me (Mangel and Clark 1988, Houston and McNamara 1999, Clark and Mangel 2000, McNamara et al. 2001). The parasitoid problem that we considered is basically one of investment (in this case, of eggs) in situations with uncertainty (in this case maternal survival). Such problems abound and the methods have a wide range of applications in economic settings. One of my favorite volumes is Dixit and Pindyck (1994). Owen-Smith (2002) gives some interesting applications of these methods to herbivore diet choice and ecology. More mathematical sources include Bertsekas (1976), Whittle (1983), Mangel (1985), Puterman (1994), and Bertsekas (1995).

0 0

Post a comment