Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Homework Help: Dynamic Programming with 2 state variables

  1. Nov 8, 2012 #1
    1. The problem statement, all variables and given/known data

    Derive the Euler Equation of the dynamic programming problem:

    [itex]\[max_{\{ z_t \}^\infty_{t=0}}
    \sum_{t=0}^{\infty} \delta^t f(x_t, y_t, z_t)
    \] [/itex]

    subject to:

    [itex] x_{t+1} = g_1(x_t, y_t, z_t),

    \

    y_{t+1} = g_2(x_t, y_t, z_t),

    \
    x_0 = x^0,
    y_0 = y^0 [/itex]

    and where [itex] \delta <1 [/itex]
    2. Relevant equations

    We can write the value function as:

    [itex]V(x,y) = max_z [f(x, y, z) + \delta V(g_1(x, y, z), g_2(x, y, z))][/itex]

    3. The attempt at a solution

    The solution is characterized by 3 equations:

    The first-order-condition at the optimal z* is:

    [itex] \frac{\partial f(x, y, z^*)}{\partial z} + \delta[\frac{\partial V (g_1(x, y, z^*), g_2(x, y, z^*))}{\partial g_1(x, y, z^*)}\frac{\partial g_1(x, y, z^*)}{\partial z} + \frac{\partial V (g_1(x, y, z^*), g_2(x, y, z^*))}{\partial g_2(x, y, z^*)}\frac{\partial g_2(x, y, z^*)}{\partial z}] = 0


    [/itex]

    Differentiating the value function with respect to each state gives us:

    [itex]

    \frac{\partial V (x, y)}{\partial x} = \frac{\partial f(x, y, z^*)}{\partial x} + \delta[\frac{\partial V (g_1(x, y, z^*), g_2(x, y, z^*))}{\partial g_1(x, y, z^*)}\frac{\partial g_1(x, y, z^*)}{\partial x} + \frac{\partial V (g_1(x, y, z^*), g_2(x, y, z^*))}{\partial g_2(x, y, z^*)}\frac{\partial g_2(x, y, z^*)}{\partial x}]

    [/itex]

    and

    [itex]

    \frac{\partial V (x, y)}{\partial y} = \frac{\partial f(x, y, z^*)}{\partial y} + \delta[\frac{\partial V (g_1(x, y, z^*), g_2(x, y, z^*))}{\partial g_1(x, y, z^*)}\frac{\partial g_1(x, y, z^*)}{\partial y} + \frac{\partial V (g_1(x, y, z^*), g_2(x, y, z^*))}{\partial g_2(x, y, z^*)}\frac{g_2(x, y, z^*)}{\partial y}]

    [/itex]

    I think I should be able to derive an Euler Equation with the above 3 equations, but I'm not sure how to manipulate the equations to get a meaningful answer.
     
    Last edited: Nov 8, 2012
  2. jcsd
  3. Nov 8, 2012 #2
    bump. any help would be greatly appreciated!
     
  4. Nov 8, 2012 #3
    bumping this on the off-chance somebody will be able to help out.
     
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook