Pontryagin minimum principle with control constraints

[synMehdi]

Hi, I am trying to solve a control problem where I have to minimize the fuel consumption of a vehicle:
$$J=\int_{0}^{T} L(x(t), u(t),t) + g(x(T),T)dt$$
$L(u(t),v(t))=\sum\limits_{i,j=0}^{2} K_{i,j} u(t)^i v(t)^j$ is convex (quadratic) and the term $g(x(T),T)$ is to have a constraint in the value of the last state - in my case $x(T)=x(0)$
Subject to some dynamic constraints on the state:
$$\dot{x}(t)=f(x(t),u(t))$$
and some inequality constraints on the control variable (which is what I'm having trouble with)
$$U_{min}<u(t)<U_{max}$$
So far I've been able to solve the unconstrained problem by minimizing the Hamiltonian
$$H(x(t),u(t),p(t))=L(x(t), u(t),t)+p(t) \dot{x(t)}$$
The co-state $p(t)=p_0$ is constant and calculated so that $x(T)=x(0)$. Also $u(t)$ is function of $p_0$ and some of the quadratic parameters that come from derivative of $L(x(t),u(t),t)$ wrt to $u(t)$ ($\frac{dH}{du}=0$)
This works fine and i have the results I want. My question is how can I incorporate the inequality constrains on the control input: $u(t)$ without losing the $p_0$ that makes $x(T)=x(0)$

 

[tnich]

Hi, I am trying to solve a control problem where I have to minimize the fuel consumption of a vehicle:
$$J=\int_{0}^{T} L(x(t), u(t),t) + g(x(T),T)dt$$
$L(u(t),v(t))=\sum\limits_{i,j=0}^{2} K_{i,j} u(t)^i v(t)^j$ is convex (quadratic) and the term $g(x(T),T)$ is to have a constraint in the value of the last state - in my case $x(T)=x(0)$
Subject to some dynamic constraints on the state:
$$\dot{x}(t)=f(x(t),u(t))$$
and some inequality constraints on the control variable (which is what I'm having trouble with)
$$U_{min}<u(t)<U_{max}$$
So far I've been able to solve the unconstrained problem by minimizing the Hamiltonian
$$H(x(t),u(t),p(t))=L(x(t), u(t),t)+p(t) \dot{x(t)}$$
The co-state $p(t)=p_0$ is constant and calculated so that $x(T)=x(0)$. Also $u(t)$ is function of $p_0$ and some of the quadratic parameters that come from derivative of $L(x(t),u(t),t)$ wrt to $u(t)$ ($\frac{dH}{du}=0$)
This works fine and i have the results I want. My question is how can I incorporate the inequality constrains on the control input: $u(t)$ without losing the $p_0$ that makes $x(T)=x(0)$

Could you create a potential well so that the potential energy rises steeply at $u = U_{max}$ and $u = U_{min}$? Something like $P(u) =\left(\frac {2u- (U_{max}+U_{min})} {U_{max}-U_{min}}\right)^{2n}$. Then you could look at the result in the limit as $n \rightarrow \infty$.

 

[tnich]

Could you create a potential well so that the potential energy rises steeply at $u = U_{max}$ and $u = U_{min}$? Something like $P(u) =\left(\frac {2u- (U_{max}+U_{min})} {U_{max}-U_{min}}\right)^{2n}$. Then you could look at the result in the limit as $n \rightarrow \infty$.

Or it might be easier to use Heaviside step functions to construct the potential well.

 

[synMehdi]

Thanks for the answer.
The solution that you propose seems difficult to include analytically.
My question is more about how the Pontryagin Minimum Principle works when whe have constraints. I suppose that $\frac{dH}{du} = 0$ is no longer valid so I will have to look in the borders ($H(U_{max})$ and $H(U_{min})$) and check if it is smaller than the u given by $\frac{dH}{du} = 0$.
In my case as as it is quadratic it will simply be $Min(Max(argmin(H),U_{min}),U_{max})$
But then how would my co-state variable $p$ change (maybe it won't be constant anymore) and how to calculate it in this case so that I don't lose the property of $x(T)=x(0)$

 

[tnich]

Thanks for the answer.
The solution that you propose seems difficult to include analytically.
My question is more about how the Pontryagin Minimum Principle works when whe have constraints. I suppose that $\frac{dH}{du} = 0$ is no longer valid so I will have to look in the borders ($H(U_{max})$ and $H(U_{min})$) and check if it is smaller than the u given by $\frac{dH}{du} = 0$.
In my case as as it is quadratic it will simply be $Min(Max(argmin(H),U_{min}),U_{max})$
But then how would my co-state variable $p$ change (maybe it won't be constant anymore) and how to calculate it in this case so that I don't lose the property of $x(T)=x(0)$

OK. I was trying to give you a term that you could add to the Hamiltonian that would express the constraint and be differentiable.

 

[tnich]

OK. I was trying to give you a term that you could add to the Hamiltonian that would express the constraint and be differentiable.

Have you tried using Lagrange multipliers?

 

[synMehdi]

Have you tried using Lagrange multipliers?

Well I thought multipliers were for equality constraints not inequality. Can they be used in this case?

OK. I was trying to give you a term that you could add to the Hamiltonian that would express the constraint and be differentiable.

I now understand what you were proposing and it is a good idea, like an infinity penalty for going outside the bounds of $u(t)$, right?. I will see at the maths of it, try it and get back.

 

[tnich]

Well I thought multipliers were for equality constraints not inequality. Can they be used in this case?

You can use Lagrange multipliers with inequalities, but you need to constrain the multiplier to be non-negative so that the product $λ(u-u_{min})$ (for example) is $\geq 0$. That is, since $u \geq u_{min}$ and $λ\geq 0$, $λ(u-u_{min}) \geq 0$. Then you add $λ(u-u_{min})$ to the function you are trying to minimize. I am not sure that really helps you.

I now understand what you were proposing and it is a good idea, like an infinity penalty for going outside the bounds of $u(t)$, right?. I will see at the maths of it, try it and get back.

Yes, that is the idea.

 

[tnich]

Well I thought multipliers were for equality constraints not inequality. Can they be used in this case?

I now understand what you were proposing and it is a good idea, like an infinity penalty for going outside the bounds of $u(t)$, right?. I will see at the maths of it, try it and get back.

I have been imagining the kinds of problems you have been running into in implementing this approach. I want to suggest an alternative algorithm:
1) Solve the unconstrained problem.
2) If constraint is violated, then re-solve the problem under the assumption that the system operates at the constraint from time $t_1$ to time $t_2$ (e.g. for $t \in [t_1,t_2]$, $u(t)=u_{min}$). You will have to do three separate optimizations (again assuming the $u_{min}$ constraint is violated):
a) For $t \in [0,t_1]$ with the constraint that $u(t_1)=u_{min}$
b) For $t \in [t_1,t_2]$ with the constraint that $u(t) = u_{min}$ is constant
c) For $t \in [t_2,t_f]$ with the constraint that $u(t_2)=u_{min}$
3) Take the result of these three optimizations and optimize over $t_1$ and $t_2$.

If the result violates the other constraint, you may need to use a similar method to deal with both constraints simultaneously.

The complexity of this approach grows exponentially in the number of constraints, so while it might work in a simple case, it would not be very good with a lot of constraints.