Computer Vision - Ground plane position based on image point

[jenny_shoars]

When given an image of a scene of something like a hallway or road (looking down this hallway or road) a vanishing point can be determined. Also, the points on the image must lie on a certain line in the real world environment given by:

$x_{im} = f\frac{X}{Z}$ and $y_{im} = f\frac{Y}{Z}$

where $f$ is the focal length of the camera, $(x_{im},y_{im})$ is the image point and $(X, Y, Z)$ is the world frame point. This is illustrated below.

I've found that if the camera optical axis is in the same direction as the hallway/road (i.e. the optical center goes to the vanishing point) and the camera is just a certain height above the ground plane, then it's simple to get the position of a given image point on the ground plane. That being, if $H$ is the height, then:
$Y=-H$, $Z=-H$, $X=-frac{x_{im}H}{y_{im}}$
This all works fine, but I'm getting a little lost when adding in yaw and pitch to the camera reference frame (no roll). After some work I've found that the ground plane can be represented in the camera reference frame by:

$X sin(\alpha) sin(\beta) + Z sin(\alpha) cos(\beta) + Y cos(\alpha) = H$

And this is where I'm stuck. I'm not sure how to usefully use the above equation with the first two equations to get $X$, $Y$, and $Z$ from $x_{im}$ and $y_{im}$. Any suggestions? Thank you much!

 

[eljose79]

Thank you for sharing your findings on determining the position of an image point on a ground plane using a camera and vanishing point. I would like to offer some suggestions on how to approach the equation you have mentioned.

Firstly, it is important to understand the different variables involved in the equation. The focal length of the camera, f, is a fixed value that determines the perspective of the image. The image point, (x_{im}, y_{im}), represents the coordinates of the point on the image where the vanishing point is located. The world frame point, (X, Y, Z), represents the real-world coordinates of the point on the ground plane.

Next, to incorporate yaw and pitch into the equation, we need to consider the rotation of the camera. Yaw refers to the rotation of the camera around the vertical axis, while pitch refers to the rotation around the horizontal axis. These rotations can be represented by the angles \alpha and \beta, respectively.

To incorporate these rotations into the equation, we can use the rotation matrix to transform the coordinates from the camera reference frame to the world frame. This transformation can be represented by the following equation:

\begin{pmatrix} X \\ Y \\ Z \end{pmatrix} = \begin{pmatrix} \cos\alpha & 0 & \sin\alpha \\ 0 & 1 & 0 \\ -\sin\alpha & 0 & \cos\alpha \end{pmatrix} \begin{pmatrix} \cos\beta & -\sin\beta & 0 \\ \sin\beta & \cos\beta & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} x_{im} \\ y_{im} \\ f \end{pmatrix}

Now, we can substitute the values of X, Y, and Z from the transformed coordinates into the equation for the ground plane:

\begin{align} X \sin\alpha \sin\beta + Z \sin\alpha \cos\beta + Y \cos\alpha &= H \\ (\cos\alpha x_{im} + \sin\alpha y_{im}) \sin\beta + (-\sin\alpha x_{im} + \cos\alpha y_{im}) \cos\beta + f \cos\alpha &= H \\ x_{im} \sin\beta \