1. Limited time only! Sign up for a free 30min personal tutor trial with Chegg Tutors
    Dismiss Notice
Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Bellman Equation, Dynamic Programming, state vs control

  1. Oct 23, 2015 #1
    Hi, I am proficient in standard dynamic programming techniques.
    In the standard textbook reference, the state variable and the control variable are separate entities.
    However, I have seen examples in economics, in which a single variable, let's say consumption, is both a state variable and a control variable simultaneously.

    This is very strange. Can the same variable be a control variable and state variable simultaneously? Is it allowed in bellman equation?
     
  2. jcsd
  3. Oct 23, 2015 #2

    jedishrfu

    Staff: Mentor

    Are you asking if say a state variable like the number of names in a list can be used as a control variable then the answer is yes.

    Code (Python):

    if nameCount>0:
       print "the name list has "+nameCount+" names."
    else:
      print "The name list is empty."
    end
     
    This style of programming is very common and is more succinct than say having a state flag:

    Code (Python):

    isNameListEmpty = true

    ...

    if nameCount>0:
      isNameListEmpty=false
    end

    ...

    if isNameListEmpty==false:
       print "the name list has "+nameCount+" names."
    else:
      print "The name list is empty."
    end
     
     
  4. Oct 23, 2015 #3
    I'm sorry this is not pertaining to my question at all. I am not asking a computer programming question. Dynamic programming is a field in mathematics.

     
  5. Oct 23, 2015 #4

    jedishrfu

    Staff: Mentor

    Here's a reference from Wikipedia on Dynamic Programming for other posters interested in the topic:

    https://en.wikipedia.org/wiki/Dynamic_programming

    This discipline is used in several areas of study including computer science. Some background history for the name:

    - excerpted from the Wikipedia article above.

    The example I gave shows a state variable # of names in a list being used as a control variable directing the algorithm to print one of two messages. Sometimes you see the second construct come up when new programmers are trying to implement an algorithm that they don't totally understand. In my example, the isNameListEmpty boolean is unnecessary since its equivalent to checking the condition nameCount==0 and if implemented incorrectly in a multi-threaded environment could result in the isNameListEmpty not matching the nameCount==0 condition.

    Here's more on the Bellman-Ford algorithm:

    https://en.wikipedia.org/wiki/Bellman–Ford_algorithm
     
    Last edited: Oct 23, 2015
  6. Oct 23, 2015 #5

    PeterDonis

    User Avatar
    2016 Award

    Staff: Mentor

    According to the Wikipedia page: "Dynamic programming is both a mathematical optimization method and a computer programming method." So it's both. But it's clear now that you are asking about the mathematical optimization method.

    Which textbook?

    Can you give a specific reference?
     
  7. Oct 23, 2015 #6

    PeterDonis

    User Avatar
    2016 Award

    Staff: Mentor

    I'm not sure this simple example really illustrates the use of "state variables" and "control variables" in the field of optimization methods. Unfortunately the online info I've been able to find does not give explicit definitions of those terms. Hopefully the OP will give us more specific references. I'm assuming that the Bellman Equation he's referring to is this:

    https://en.wikipedia.org/wiki/Bellman_equation
     
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook




Similar Discussions: Bellman Equation, Dynamic Programming, state vs control
  1. Equation vs function (Replies: 3)

Loading...