Reproducing Optimal Cost in Linear Quadratic Regulator Problem

  • Context: Graduate 
  • Thread starter Thread starter matematikawan
  • Start date Start date
  • Tags Tags
    Control Optimal control
Click For Summary

Discussion Overview

The discussion revolves around the challenges faced in reproducing the optimal cost in the linear quadratic regulator (LQR) problem as presented in a specific paper. Participants explore various methods for solving the problem, including Pontryagin's Minimum Principle and the Riccati equation, while addressing issues related to control variables, terminal conditions, and system stability.

Discussion Character

  • Exploratory
  • Technical explanation
  • Debate/contested
  • Mathematical reasoning

Main Points Raised

  • One participant is attempting to reproduce the optimal cost for an LQR problem but is obtaining a different result than expected, suggesting a possible misunderstanding of concepts.
  • Another participant questions the assumption that the control variable \( u \) is a scalar, proposing that it should be treated as a vector, which affects the Hamiltonian formulation.
  • There is a discussion about the initial and terminal conditions, with one participant noting the lack of a specified terminal point in the problem.
  • Participants discuss the distinction between fixed end and free end terminal problems, with some expressing confusion about how to categorize the problem they are working on.
  • One participant mentions that the paper describes the system as a "linear unstable system," prompting questions about the implications of instability on the computation.
  • Another participant suggests that further research, such as searching Google Scholar, may provide clarity on the concept of linear instability.

Areas of Agreement / Disagreement

Participants express differing views on the nature of the control variable and the treatment of terminal conditions. There is no consensus on the implications of the system being unstable, nor on how to approach the problem regarding terminal conditions.

Contextual Notes

Participants note the importance of understanding the initial and terminal conditions in optimal control problems, as well as the potential impact of system stability on the results. There are unresolved questions regarding the correct formulation of the Hamiltonian and the classification of the terminal problem type.

Who May Find This Useful

This discussion may be useful for individuals studying optimal control theory, particularly those interested in the linear quadratic regulator problem and its associated mathematical formulations.

matematikawan
Messages
336
Reaction score
0
I'm trying to pick up optimal control by self study. At the moment I'm working on linear quadratic regulator and trying to reproduce the result publish in this paper.
Curtis and Beard, Successive collocation: An approximation to optimal nonlinear control, Proceedings of the American Control Conference 2001.

The problem is:
Minimize J(x)=\int_0^{10} x^Tx + u^Tu dt
subject to
\dot{x}=Ax+Bu; \ \ x_0^T=(-12,20)
where
A=\left(\begin{array}{cc}0&1\\-1&2\end{array}\right)
B=\left(\begin{array}{cc}0\\1\end{array}\right)

Answer for optimal cost is J*(x)=2221.

However I have try a few times but cannot reproduce this answer. I obtain 2346.5 instead using the methods of Pontryagin's Minimum Principle or Riccati equation. Probably I have misunderstood some concept here.

Using Pontryagin's Minimum Principle, I let the Hamiltonian
H=x_1^2 + x_2^2 + u^2 + \lambda_1x_2 + \lambda_2(-x_1+2x_2+u)

From which I can obtain 5 equations.

\dot{x}=Ax+Bu
\dot{\lambda}_1 = -\frac{\partial H}{\partial x_1}
\dot{\lambda}_2 = -\frac{\partial H}{\partial x_2}
\frac{\partial H}{\partial u}=0
This linear system can be solve subject to the conditions
x_1(0)=-12, x_2(0)=20, \lambda_1(10)=0 , \lambda_2(10)=0.

The solutions are plug into
J(x)=\int_0^{10} x^Tx + u^Tu dt.

Any clue where did I gone wrong? Or do anybody know a program that can compute the answer. I know there is a MATLAB command lqr but it only gives the feedback control not the value of the optimal cost.
 
Physics news on Phys.org
How do you know u is a scalar? from the problem u the control variable is a vector. Thus, your Hamiltonian is wrong.

H = x_{1}^{2} + x_{2}^{2} + u_{1}^{2} + u_{2}^2 + \vec{\lambda}^{T} (Ax + Bu)

Also you forgot to say anything about the initial and terminal conditions...
 
Thanks Pyrrhus. Probably that's my mistake.

My argument why the control u is a scalar because in the equation \dot{x}=Ax+Bu , B is a column vector. The only way we can compute Bu is when u is a scalar.

Bu=\left(\begin{array}{cc}0\\u\end{array}\right).

The initial condition x(0) is specified as x1(0)=-12, x2(0)=20,
but the terminal point x(T) is not given.
 
Ok, it makes sense.

Did you try solving it as a free end terminal problem?

It looks like you solved as a fixed end terminal problem.
 
Pyrrhus said:
Did you try solving it as a free end terminal problem?

It looks like you solved as a fixed end terminal problem.


This is the part that really confuse me, the terminal point, because so far I have been doing by just following examples.

Some problem have specific fixed end. Whilst others are free and yet some have infinite time.
So I'm not fully understand what I'm doing here whether it is fixed end, free end or infinite time.

I guess I'm solving it as a free terminal point because I'm taking the costate value at terminal point as zero, \lambda_1(T)=\lambda_2(T)=0.
 
matematikawan said:
This is the part that really confuse me, the terminal point, because so far I have been doing by just following examples.

Some problem have specific fixed end. Whilst others are free and yet some have infinite time.
So I'm not fully understand what I'm doing here whether it is fixed end, free end or infinite time.

I guess I'm solving it as a free terminal point because I'm taking the costate value at terminal point as zero, \lambda_1(T)=\lambda_2(T)=0.

This is important. I'd recommend reading the paper and identifying the initial and final conditions.
 
I have gone through the paper again but cannot extract new information about the terminal point other than what I have already written.

But I see there is a sentence which claim that this example is for linear unstable system.
Why is it unstable? Will it effect the computation?
 
matematikawan said:
I have gone through the paper again but cannot extract new information about the terminal point other than what I have already written.

But I see there is a sentence which claim that this example is for linear unstable system.
Why is it unstable? Will it effect the computation?

That's a good question. I am not sure what "linear unstable system" means.

I know Dynamic Optimization, because economists use the theory. I am not an Electronic/Electric Engineer, so I am not sure.

Perhaps a search on Google Scholar will help you?
 

Similar threads

  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 3 ·
Replies
3
Views
3K
  • · Replies 8 ·
Replies
8
Views
3K
  • · Replies 0 ·
Replies
0
Views
4K
  • · Replies 8 ·
Replies
8
Views
2K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 3 ·
Replies
3
Views
4K
  • · Replies 61 ·
3
Replies
61
Views
13K
  • · Replies 1 ·
Replies
1
Views
6K
  • · Replies 9 ·
Replies
9
Views
2K