Inverse perspective mapping equations

In summary: The key is to find the transformation matrix that will map the pixel coordinates (xp, yp) to real world coordinates (X, Y). This can be done by calibrating the camera and using a grid pattern on the ground to determine the mapping. Alternatively, it can be done theoretically by accounting for the visual projection at the focal point, the distortion caused by the lens, and the angle of the camera.In summary, the conversation discusses how to map points in the real world to points on an image captured by a camera. It is suggested to use a grid pattern and calibrate the camera, or to find the transformation matrix theoretically by considering the lens distortion and camera angle. The parameters needed for this mapping include the focal length, height,
  • #1
darie2808
2
0
Hi . I am making a robot that is supposed to detect a red ball with a camera, and then know where the ball is .
The robot has a camera on top of it, that is tilted at an angle ; At each frame the camera detects the red ball and returns 2 coordinates , xp and yp which are the pixel coordinates of the middle of the ball on the image captured .
What i want is to map each point on the ground to a point in the image. In other words to change the perspective from birds eye view to a 2d top view of the ground in front of the robot.
Everything about the camera is known : focal length , the angle at which is tilted, the height at which it is , the horisontal and vertical apperture angles...everything.
All i need is the formula that relates X,Y of the real world to the xp and yp .
I tried several equations that i found online but although all are similar, they are different and none worked .
Note the (0,0) point on the image is in the middle of it.
Help would be much appreciated!
Best wishes ,
Darie
 
Physics news on Phys.org
  • #2
I've worked with a group that had to solve the same problem. I didn't work on that part of the project but my recollection is that for the most part what we did was to mark off the floor in a grid pattern. We then calibrated the camera by taking a picture of the grid on the floor and numerically working out the mapping. That may be one of the better ways of doing it (particularly if you end up changing cameras or simply adjusting the rig as time goes by). This worked quite well actually because at the time we were making a point of using very cheap web cameras which presented an appreciable amount of distortion.

Now if you wish do this purely from theory, I think the first starting point is coming up with a mapping for a perfect lens. That is, the first thing you need is to account for the visual projection at the focal point of the floor space. This would only depend upon the height of the camera. You would probably then have to derive a projection for the distortion that is caused by the lens (this is could be captured empirically like we did as I described above). My guess is that you would most likely need to accommodate just for simple barrel distortion. Once you come up with these projections you would simply need to apply the inverse projections on the resulting capture image (image * inverse barrel distortion * inverse perspective projection).
 
  • #3
Thank you for your answer! We might do the same as you did . However our camera has very low resolution (174*144) . Hope it will work . But there is a problem with that, if we change the angle of the camera, all the mapping will have to be done again ... Right? I might have missunderstood you though , since you said that that changing the rig is not a problem .
What are the parameters that we really need ? Focal length , height , camera angle and xp and yp right? Do we need anything else? Like the aperture angle? or something like that.
All the equations we encountered required different parameters.
Again , what I need is the 'transformation matrix ' such that :
(X,Y) = transformation matrix * (xp,yp).
DO you know where I could find this transformation matrix?
 
  • #4
Well ideally you would just write an automated procedure to calibrate the rig. That way, you can setup your calibration grid on the floor and then just run the calibration procedure any time you wish. With such a program you would not need to input the parameters like focal length, height, etc. Plus one would expect some slight deviation from the ideal when put into practice anyway and thus there should be some kind of empircal calibration in the end.

As for the transformation matrix, no I don't know how to do that with a lens. I know how to do that assuming that you do not have a lens easily enough but I do not remember enough of my perspective and projection lessons to recall how you would accommodate the added refraction of a lens between the picture plane or focal point and the object. The problem is that the ray tracing involves skew lines instead of lines that are parallel to the axis. You can probably find more about this in Born and Wolf's "Principles of Optics" text. I believe they cover the behavior of skew lines in 4.9. But you should be able to find the transformation for basic distortions easily enough.

Either way, I would take a look at Born and Wolf's text or even take a look at some engineering drafting textbooks. Techniques on how to do technical perspective drawings may be more applicable in this case since you want to do a rough ray trace based method.

EDIT: Off-hand here is one way how I would think of the problem but I do not know if it is right. I would just do a projection drawing of the grid given the height and angle of the camera and the distance from the grid. I would do a projection onto a picture plane that is of the focal length away from the focal point and the picture plane being situated in a plane dictated by the angle of the camera. I'm not sure if this is correct but it sounds reasonable to me. This would give you the basic projection results to which the lens aberrations and distortions would be applied to.
 
  • #5
This was my idea on how it is probably done but I'm not sure.
 

Attachments

  • method.jpg
    method.jpg
    33.1 KB · Views: 727

Related to Inverse perspective mapping equations

What is inverse perspective mapping (IPM)?

Inverse perspective mapping (IPM) is a mathematical process used to correct the distortion in images captured by a camera with a wide field of view. It works by transforming the image into a bird's-eye view perspective, making it easier to analyze and extract information from the image.

Why is IPM used in computer vision?

IPM is used in computer vision to improve image analysis and object detection. By correcting the distortion in images, IPM provides a more accurate representation of the scene, making it easier for algorithms to identify and locate objects. This is particularly useful in applications such as autonomous driving and surveillance.

How does IPM work?

IPM works by using a set of mathematical equations to transform the image into a bird's-eye view perspective. These equations take into account the camera's intrinsic parameters (such as focal length and lens distortion) and the camera's position and orientation in the scene. The result is a perspective-corrected image that can be analyzed more accurately.

What are the limitations of IPM?

IPM can only be applied to images captured by cameras with a wide field of view, such as fisheye lenses. It also requires accurate knowledge of the camera's intrinsic parameters and position, which may not always be available. Additionally, IPM can be computationally intensive, making it less suitable for real-time applications.

Can IPM be used for any type of image?

IPM is mainly used for images captured by cameras with a wide field of view, but it can also be applied to other types of images, such as satellite images and aerial photographs. However, the effectiveness of IPM may vary depending on the image resolution and quality.

Similar threads

  • Classical Physics
Replies
21
Views
1K
  • Differential Geometry
Replies
1
Views
3K
  • Introductory Physics Homework Help
Replies
2
Views
1K
  • Introductory Physics Homework Help
Replies
14
Views
2K
  • Biology and Medical
Replies
2
Views
3K
Replies
3
Views
3K
  • General Discussion
Replies
3
Views
4K
  • Special and General Relativity
Replies
6
Views
1K
  • Astronomy and Astrophysics
Replies
34
Views
12K
  • Other Physics Topics
Replies
2
Views
3K
Back
Top