At the present level of technology, video is not coded in terms of objects. It's just a flat rectangular picture and a set of rules that prescribe how to construct frame #N+1 from frames #N, #N-1, etc. The best you can do is calculate the speed of movement of some point of the video, in pixels per second. You can't backtrack from that to physical movement. Most of the time, if you want physical movement, you need videos from two cameras with predetermined distance between them, and predetermined optical properties such as field of view angle and optical distortion characteristics (though there are tricks that will let you deduce field of view and optical distortion from the video).
Things you see in movies often have no relation to the state of technology in the real world. It always gets me in movies and detective shows, when "technicians" take a grainy 320x240 shot from an ATM camera and use it to zoom in onto a license plate in the background. CSI and Law & Order are often guilty of that. I actually work on digital video for a living and I like to think that I know what can or can't be done.
There's been a shift towards three-dimensional video recently (heard of Avatar?) and, with that kind of video (which is basically two parallel tracks from two different fixed cameras), you can do some 3D reconstruction using 3D algorithms. Even then it's not easy. There's an open-source library called OpenCV, but I haven't personally worked with that one. In my opinion, 10 years from now there will be a universally adopted true 3D standard, but, for now, that work is in its infancy.