1. Not finding help here? Sign up for a free 30min tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Dynamic Programming with 2 state variables

  1. Nov 8, 2012 #1
    1. The problem statement, all variables and given/known data

    Derive the Euler Equation of the dynamic programming problem:

    [itex]\[max_{\{ z_t \}^\infty_{t=0}}
    \sum_{t=0}^{\infty} \delta^t f(x_t, y_t, z_t)
    \] [/itex]

    subject to:

    [itex] x_{t+1} = g_1(x_t, y_t, z_t),

    \

    y_{t+1} = g_2(x_t, y_t, z_t),

    \
    x_0 = x^0,
    y_0 = y^0 [/itex]

    and where [itex] \delta <1 [/itex]
    2. Relevant equations

    We can write the value function as:

    [itex]V(x,y) = max_z [f(x, y, z) + \delta V(g_1(x, y, z), g_2(x, y, z))][/itex]

    3. The attempt at a solution

    The solution is characterized by 3 equations:

    The first-order-condition at the optimal z* is:

    [itex] \frac{\partial f(x, y, z^*)}{\partial z} + \delta[\frac{\partial V (g_1(x, y, z^*), g_2(x, y, z^*))}{\partial g_1(x, y, z^*)}\frac{\partial g_1(x, y, z^*)}{\partial z} + \frac{\partial V (g_1(x, y, z^*), g_2(x, y, z^*))}{\partial g_2(x, y, z^*)}\frac{\partial g_2(x, y, z^*)}{\partial z}] = 0


    [/itex]

    Differentiating the value function with respect to each state gives us:

    [itex]

    \frac{\partial V (x, y)}{\partial x} = \frac{\partial f(x, y, z^*)}{\partial x} + \delta[\frac{\partial V (g_1(x, y, z^*), g_2(x, y, z^*))}{\partial g_1(x, y, z^*)}\frac{\partial g_1(x, y, z^*)}{\partial x} + \frac{\partial V (g_1(x, y, z^*), g_2(x, y, z^*))}{\partial g_2(x, y, z^*)}\frac{\partial g_2(x, y, z^*)}{\partial x}]

    [/itex]

    and

    [itex]

    \frac{\partial V (x, y)}{\partial y} = \frac{\partial f(x, y, z^*)}{\partial y} + \delta[\frac{\partial V (g_1(x, y, z^*), g_2(x, y, z^*))}{\partial g_1(x, y, z^*)}\frac{\partial g_1(x, y, z^*)}{\partial y} + \frac{\partial V (g_1(x, y, z^*), g_2(x, y, z^*))}{\partial g_2(x, y, z^*)}\frac{g_2(x, y, z^*)}{\partial y}]

    [/itex]

    I think I should be able to derive an Euler Equation with the above 3 equations, but I'm not sure how to manipulate the equations to get a meaningful answer.
     
    Last edited: Nov 8, 2012
  2. jcsd
  3. Nov 8, 2012 #2
    bump. any help would be greatly appreciated!
     
  4. Nov 8, 2012 #3
    bumping this on the off-chance somebody will be able to help out.
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook




Similar Discussions: Dynamic Programming with 2 state variables
Loading...