You have three videos, one labeled before.avi, another kalman.avi, and the third "Visual ModelBased Vehicle Tracking using an Extended Kalman Filter". That third one with the long title looks pretty good. Is this third one not your work, something to show that the problem is tractable?
Anyhow, the first one looks like you have some Kalmanlike methodology going on as the noise is strongest when the vehicle is smallish (further away from the camera) and is turning. There appears to be a good lock in that section where the vehicle is going straight.
The second one is obviously the problem. There is a basic problem with the EKF, and you may have run into it. When the filtered solution is far from the truth (and hence far from the measurements) the filter can diverge. You may need to adjust your tuning parameters. Take in a bit more or bit less measurement than your sensor model suggests, for example. Sometimes you have to lie to the filter about the performance of your sensors. Or lie to the filter about plant noise. Or both. Filter tuning can be a bit more art than science.
It also looks like you may have some problems with your state descriptions or your derivatives. The goofy behavior always appears to start with a step change in the filtered solution. It then rotates for a short while, makes another step change, rotates for a bit, ..., before finally locking on to something close to the truth. Are you using Euler angles for rotational state? If so, you may be running into a singularity/gimbal lock.
