Bellman Equation, Dynamic Programming, state vs control

[onthetopo]

Hi, I am proficient in standard dynamic programming techniques.
In the standard textbook reference, the state variable and the control variable are separate entities.
However, I have seen examples in economics, in which a single variable, let's say consumption, is both a state variable and a control variable simultaneously.

This is very strange. Can the same variable be a control variable and state variable simultaneously? Is it allowed in bellman equation?

 

[jedishrfu]

Are you asking if say a state variable like the number of names in a list can be used as a control variable then the answer is yes.

if nameCount>0:
   print "the name list has "+nameCount+" names."
else:
  print "The name list is empty."
end

This style of programming is very common and is more succinct than say having a state flag:

isNameListEmpty = true

...

if nameCount>0:
  isNameListEmpty=false
end

...

if isNameListEmpty==false:
   print "the name list has "+nameCount+" names."
else:
  print "The name list is empty."
end

 

[onthetopo]

I'm sorry this is not pertaining to my question at all. I am not asking a computer programming question. Dynamic programming is a field in mathematics.

Are you asking if say a state variable like the number of names in a list can be used as a control variable then the answer is yes.

if nameCount>0:
   print "the name list has "+nameCount+" names."
else:
  print "The name list is empty."
end

This style of programming is very common and is more succinct than say having a state flag:

isNameListEmpty = true

...

if nameCount>0:
  isNameListEmpty=false
end

...

if isNameListEmpty==false:
   print "the name list has "+nameCount+" names."
else:
  print "The name list is empty."
end

 

[jedishrfu]

Here's a reference from Wikipedia on Dynamic Programming for other posters interested in the topic:

https://en.wikipedia.org/wiki/Dynamic_programming

This discipline is used in several areas of study including computer science. Some background history for the name:

History
The term dynamic programming was originally used in the 1940s by Richard Bellman to describe the process of solving problems where one needs to find the best decisions one after another. By 1953, he refined this to the modern meaning, referring specifically to nesting smaller decision problems inside larger decisions,[16] and the field was thereafter recognized by the IEEE as a systems analysis and engineering topic. Bellman's contribution is remembered in the name of the Bellman equation, a central result of dynamic programming which restates an optimization problem in recursive form.

Bellman explains the reasoning behind the term dynamic programming in his autobiography, Eye of the Hurricane: An Autobiography (1984). He explains:

"I spent the Fall quarter (of 1950) at RAND. My first task was to find a name for multistage decision processes. An interesting question is, Where did the name, dynamic programming, come from? The 1950s were not good years for mathematical research. We had a very interesting gentleman in Washington named Wilson. He was Secretary of Defense, and he actually had a pathological fear and hatred of the word research. I’m not using the term lightly; I’m using it precisely. His face would suffuse, he would turn red, and he would get violent if people used the term research in his presence. You can imagine how he felt, then, about the term mathematical. The RAND Corporation was employed by the Air Force, and the Air Force had Wilson as its boss, essentially. Hence, I felt I had to do something to shield Wilson and the Air Force from the fact that I was really doing mathematics inside the RAND Corporation. What title, what name, could I choose? In the first place I was interested in planning, in decision making, in thinking. But planning, is not a good word for various reasons. I decided therefore to use the word “programming”. I wanted to get across the idea that this was dynamic, this was multistage, this was time-varying. I thought, let's kill two birds with one stone. Let's take a word that has an absolutely precise meaning, namely dynamic, in the classical physical sense. It also has a very interesting property as an adjective, and that it's impossible to use the word dynamic in a pejorative sense. Try thinking of some combination that will possibly give it a pejorative meaning. It's impossible. Thus, I thought dynamic programming was a good name. It was something not even a Congressman could object to. So I used it as an umbrella for my activities."
The word dynamic was chosen by Bellman to capture the time-varying aspect of the problems, and because it sounded impressive.[11] The wordprogramming referred to the use of the method to find an optimal program, in the sense of a military schedule for training or logistics. This usage is the same as that in the phrases linear programming and mathematical programming, a synonym for mathematical optimization.[17]

- excerpted from the Wikipedia article above.

The example I gave shows a state variable # of names in a list being used as a control variable directing the algorithm to print one of two messages. Sometimes you see the second construct come up when new programmers are trying to implement an algorithm that they don't totally understand. In my example, the isNameListEmpty boolean is unnecessary since its equivalent to checking the condition nameCount==0 and if implemented incorrectly in a multi-threaded environment could result in the isNameListEmpty not matching the nameCount==0 condition.

Here's more on the Bellman-Ford algorithm:

https://en.wikipedia.org/wiki/Bellman–Ford_algorithm

 

[PeterDonis]

Dynamic programming is a field in mathematics.

According to the Wikipedia page: "Dynamic programming is both a mathematical optimization method and a computer programming method." So it's both. But it's clear now that you are asking about the mathematical optimization method.

In the standard textbook reference

Which textbook?

I have seen examples in economics, in which a single variable, let's say consumption, is both a state variable and a control variable simultaneously.

Can you give a specific reference?

 

[PeterDonis]

The example I gave shows a state variable # of names in a list being used as a control variable directing the algorithm to print one of two messages.

I'm not sure this simple example really illustrates the use of "state variables" and "control variables" in the field of optimization methods. Unfortunately the online info I've been able to find does not give explicit definitions of those terms. Hopefully the OP will give us more specific references. I'm assuming that the Bellman Equation he's referring to is this:

https://en.wikipedia.org/wiki/Bellman_equation