How Can Neural Networks Control an Inverted Pendulum on a Cart?

[runway]

Dear colleagues,

I am trying to program a java application which should demonstrate
the control of the inverse pendulum on a cart using a complex of two
neural
networks using two-stage learning process. The problem is keeping the
balance
of the rod of the pendulum.

I have found this website which thorougly describes the problem of the
inverse
pendulum on a cart:

http://gershwin.ens.fr/vdaniel/Doc-Locale/Cours-Mirrored/Methodes-Maths/white/sdyn/s7/s7invp1/s7invp1.html
http://gershwin.ens.fr/vdaniel/Doc-Locale/Cours-Mirrored/Methodes-Maths/white/sdyn/s7/s7invp2/s7invp2.html

Equations (7.64) and (7.65) should give a complete state space
representation
of the nonlinear inverted pendulum.

Equations (7.55) and (7.57)

m*x''*cos(theta) + m*l*theta'' = m*g*sin(theta) (7.57)
(M+m)*x'' - m*l*sin(theta)*(theta')^2 + m*l*cos(theta)*theta'' = u (7.55)

define this system according to the figure.

I've managed to derive those equations but I don't know at all how to
use them to compute what force must I apply to the cart in order to keep the balance of the pendulum when theta != 0. (non zero - then the pendulim is not balanced).

Is there any method to solve those equations when initial conditions are given?

What force 'u' do I need to apply to the cart when the angle between the axis perpendicular to the cart and the rod of the pendulum is theta?

Thank you for any hints.

Tomas Selnekovic

 

[zeta101]

This isn't directly answering your question but it may help at some point. Something I read once about balance is that it's not about making tiny corrections so you are always at the equilibrium position (i.e., in your case when theta = 0), it's about gently oscillating back and forth about the equilibrium position.

This works because your average postion is on the equilibium position I would imagine.

 

[pervect]

Here is a general hint.

There are many different algorithms that could in theory balance the system. The crudest algorithm is straight proportional controller. This controller would make the force directly proportional to the error angle theta.

F = k*error

A more general controller is the proportional-intergal controller.

$$F = k1*error + k2 \int error*dt$$

Sometimes a proportional-intergal-derivative controller is used

$$F = k1*error + k2 \int error*dt + k3 \frac{d error}{dt}$$

Your job is to pick a controller type, then find the values of k (or k1 & k2 - or even k1,k2, and k3, depending on your choice) that make the system stable.

A system is stable if its repsonse to errors decreases with time. This means that all the real eigenvalues of the closed loop system resonse must be real and negative.

If you don't know how to calculate the closed loop response of a system consisting of the cart plus a controller, you might want to study control theory.

You may not need to learn control theory though - if you are supposed to build a "learning box", then it should be able to "learn" the right values of the constants to make the system stable, as long as you can simulate the response of the system. You have the equations to simulate the cart already, the equations needed to simulate the PID controller are very simple and have been given in this post.

You can also try a websearch for "proportional intergal" or "proportional intergal derivative" controller.

Usually systems that compute derivatives are rather noisy, so that a proportional-intergal controler is the usual type of controller that's used. I'd only go to the "D" part of the controller if there are severe problems with a PI controller.

That's just a herustic guess, though.