# Bellman Equation, Dynamic Programming, state vs control

Tags:
1. Oct 23, 2015

### onthetopo

Hi, I am proficient in standard dynamic programming techniques.
In the standard textbook reference, the state variable and the control variable are separate entities.
However, I have seen examples in economics, in which a single variable, let's say consumption, is both a state variable and a control variable simultaneously.

This is very strange. Can the same variable be a control variable and state variable simultaneously? Is it allowed in bellman equation?

2. Oct 23, 2015

### Staff: Mentor

Are you asking if say a state variable like the number of names in a list can be used as a control variable then the answer is yes.

Code (Python):

if nameCount>0:
print "the name list has "+nameCount+" names."
else:
print "The name list is empty."
end

This style of programming is very common and is more succinct than say having a state flag:

Code (Python):

isNameListEmpty = true

...

if nameCount>0:
isNameListEmpty=false
end

...

if isNameListEmpty==false:
print "the name list has "+nameCount+" names."
else:
print "The name list is empty."
end

3. Oct 23, 2015

### onthetopo

I'm sorry this is not pertaining to my question at all. I am not asking a computer programming question. Dynamic programming is a field in mathematics.

4. Oct 23, 2015

### Staff: Mentor

Here's a reference from Wikipedia on Dynamic Programming for other posters interested in the topic:

https://en.wikipedia.org/wiki/Dynamic_programming

This discipline is used in several areas of study including computer science. Some background history for the name:

- excerpted from the Wikipedia article above.

The example I gave shows a state variable # of names in a list being used as a control variable directing the algorithm to print one of two messages. Sometimes you see the second construct come up when new programmers are trying to implement an algorithm that they don't totally understand. In my example, the isNameListEmpty boolean is unnecessary since its equivalent to checking the condition nameCount==0 and if implemented incorrectly in a multi-threaded environment could result in the isNameListEmpty not matching the nameCount==0 condition.

Here's more on the Bellman-Ford algorithm:

https://en.wikipedia.org/wiki/Bellman–Ford_algorithm

Last edited: Oct 23, 2015
5. Oct 23, 2015

### Staff: Mentor

According to the Wikipedia page: "Dynamic programming is both a mathematical optimization method and a computer programming method." So it's both. But it's clear now that you are asking about the mathematical optimization method.

Which textbook?

Can you give a specific reference?

6. Oct 23, 2015

### Staff: Mentor

I'm not sure this simple example really illustrates the use of "state variables" and "control variables" in the field of optimization methods. Unfortunately the online info I've been able to find does not give explicit definitions of those terms. Hopefully the OP will give us more specific references. I'm assuming that the Bellman Equation he's referring to is this:

https://en.wikipedia.org/wiki/Bellman_equation