Reproducing Optimal Cost in Linear Quadratic Regulator Problem

In summary, the author is trying to pick up optimal control by self study, but is having trouble reproducing a result from a paper they have read. They are using a Hamiltonian and doing Pontryagin's Minimum Principle to solve for the five equations, but are having trouble getting the answer right.
  • #1
matematikawan
338
0
I'm trying to pick up optimal control by self study. At the moment I'm working on linear quadratic regulator and trying to reproduce the result publish in this paper.
Curtis and Beard, Successive collocation: An approximation to optimal nonlinear control, Proceedings of the American Control Conference 2001.

The problem is:
Minimize [tex]J(x)=\int_0^{10} x^Tx + u^Tu dt[/tex]
subject to
[tex]\dot{x}=Ax+Bu; \ \ x_0^T=(-12,20)[/tex]
where
[tex]A=\left(\begin{array}{cc}0&1\\-1&2\end{array}\right) [/tex]
[tex]B=\left(\begin{array}{cc}0\\1\end{array}\right) [/tex]

Answer for optimal cost is J*(x)=2221.

However I have try a few times but cannot reproduce this answer. I obtain 2346.5 instead using the methods of Pontryagin's Minimum Principle or Riccati equation. Probably I have misunderstood some concept here.

Using Pontryagin's Minimum Principle, I let the Hamiltonian
[tex]H=x_1^2 + x_2^2 + u^2 + \lambda_1x_2 + \lambda_2(-x_1+2x_2+u)[/tex]

From which I can obtain 5 equations.

[tex]\dot{x}=Ax+Bu [/tex]
[tex]\dot{\lambda}_1 = -\frac{\partial H}{\partial x_1}[/tex]
[tex]\dot{\lambda}_2 = -\frac{\partial H}{\partial x_2}[/tex]
[tex]\frac{\partial H}{\partial u}=0[/tex]
This linear system can be solve subject to the conditions
[itex]x_1(0)=-12, x_2(0)=20, \lambda_1(10)=0 , \lambda_2(10)=0.[/itex]

The solutions are plug into
[tex]J(x)=\int_0^{10} x^Tx + u^Tu dt [/tex].

Any clue where did I gone wrong? Or do anybody know a program that can compute the answer. I know there is a MATLAB command lqr but it only gives the feedback control not the value of the optimal cost.
 
Physics news on Phys.org
  • #2
How do you know u is a scalar? from the problem u the control variable is a vector. Thus, your Hamiltonian is wrong.

[itex] H = x_{1}^{2} + x_{2}^{2} + u_{1}^{2} + u_{2}^2 + \vec{\lambda}^{T} (Ax + Bu) [/itex]

Also you forgot to say anything about the initial and terminal conditions...
 
  • #3
Thanks Pyrrhus. Probably that's my mistake.

My argument why the control u is a scalar because in the equation [itex]\dot{x}=Ax+Bu[/itex] , B is a column vector. The only way we can compute Bu is when u is a scalar.

[tex]Bu=\left(\begin{array}{cc}0\\u\end{array}\right).[/tex]

The initial condition x(0) is specified as x1(0)=-12, x2(0)=20,
but the terminal point x(T) is not given.
 
  • #4
Ok, it makes sense.

Did you try solving it as a free end terminal problem?

It looks like you solved as a fixed end terminal problem.
 
  • #5
Pyrrhus said:
Did you try solving it as a free end terminal problem?

It looks like you solved as a fixed end terminal problem.


This is the part that really confuse me, the terminal point, because so far I have been doing by just following examples.

Some problem have specific fixed end. Whilst others are free and yet some have infinite time.
So I'm not fully understand what I'm doing here whether it is fixed end, free end or infinite time.

I guess I'm solving it as a free terminal point because I'm taking the costate value at terminal point as zero, [itex]\lambda_1(T)=\lambda_2(T)=0[/itex].
 
  • #6
matematikawan said:
This is the part that really confuse me, the terminal point, because so far I have been doing by just following examples.

Some problem have specific fixed end. Whilst others are free and yet some have infinite time.
So I'm not fully understand what I'm doing here whether it is fixed end, free end or infinite time.

I guess I'm solving it as a free terminal point because I'm taking the costate value at terminal point as zero, [itex]\lambda_1(T)=\lambda_2(T)=0[/itex].

This is important. I'd recommend reading the paper and identifying the initial and final conditions.
 
  • #7
I have gone through the paper again but cannot extract new information about the terminal point other than what I have already written.

But I see there is a sentence which claim that this example is for linear unstable system.
Why is it unstable? Will it effect the computation?
 
  • #8
matematikawan said:
I have gone through the paper again but cannot extract new information about the terminal point other than what I have already written.

But I see there is a sentence which claim that this example is for linear unstable system.
Why is it unstable? Will it effect the computation?

That's a good question. I am not sure what "linear unstable system" means.

I know Dynamic Optimization, because economists use the theory. I am not an Electronic/Electric Engineer, so I am not sure.

Perhaps a search on Google Scholar will help you?
 

What is an Optimal Control Problem?

An Optimal Control Problem (OCP) is a mathematical optimization problem that involves finding the best control strategy for a given system. The goal of an OCP is to minimize a cost function subject to constraints while also ensuring that the system meets certain performance criteria.

What is LQR?

LQR stands for Linear-Quadratic Regulator and is a control design technique used to solve optimal control problems. It involves finding a feedback control law that minimizes a quadratic cost function, subject to linear dynamics and constraints.

What are the key components of an LQR problem?

The key components of an LQR problem are the state dynamics, control inputs, cost function, and constraints. The state dynamics describe how the system evolves over time, the control inputs are the actions taken to influence the system, the cost function quantifies the performance of the system, and the constraints limit the feasible solutions.

How is an LQR problem solved?

An LQR problem is solved by first linearizing the system dynamics around an operating point and then solving the associated algebraic Riccati equation (ARE). The solution to the ARE provides the optimal feedback control law, which can then be used to compute the optimal control inputs for the system.

What are the applications of LQR?

LQR has a wide range of applications, including but not limited to, robotics, aerospace, automotive, and process control. It is commonly used to design controllers for systems with linear dynamics and quadratic cost functions. LQR is also a key component in the field of optimal control and is often used as a building block for more complex control designs.

Similar threads

  • Differential Equations
Replies
8
Views
2K
  • Differential Equations
Replies
3
Views
2K
  • Differential Equations
Replies
1
Views
2K
Replies
8
Views
1K
  • Math Proof Training and Practice
2
Replies
61
Views
9K
  • Set Theory, Logic, Probability, Statistics
Replies
4
Views
2K
  • Engineering and Comp Sci Homework Help
Replies
1
Views
2K
  • MATLAB, Maple, Mathematica, LaTeX
Replies
1
Views
2K
  • Math Proof Training and Practice
Replies
28
Views
5K
  • Math Proof Training and Practice
6
Replies
175
Views
20K
Back
Top