Homography (computer vision)

3D plane to plane equation

We have two cameras a and b, looking at points ${\displaystyle P_i}$ in a plane. Passing from the projection ${\displaystyle {}^b{p}_i=\left({}^b{u}_i;{}^b{v}_i;1\right)}$ of ${\displaystyle P_i}$ in b to the projection ${\displaystyle {}^a{p}_i=\left({}^a{u}_i;{}^a{v}_i;1\right)}$ of ${\displaystyle P_i}$ in a:

${\displaystyle {}^a{p}_i=\frac{{}^b{z}_i}{{}^a{z}_i}{K}_a\cdot H_{ab}\cdot K_b^{-1}\cdot {}^b{p}_i}$

where ${\displaystyle {}^a{z}_i}$ and ${\displaystyle {}^b{z}_i}$ are the z coordinates of P in each camera frame and where the homography matrix ${\displaystyle H_{ab}}$ is given by

${\displaystyle H_{ab}=R-\frac{t{n}^T}{d}}$.

${\displaystyle R}$ is the rotation matrix by which b is rotated in relation to a; t is the translation vector from a to b; n and d are the normal vector of the plane and the distance from origin to the plane respectively. K_a and K_b are the cameras' intrinsic parameter matrices.

The figure shows camera b looking at the plane at distance d. Note: From above figure, assuming ${\displaystyle n^T{P}_i+d=0}$ as plane model, ${\displaystyle n^T{P}_i}$ is the projection of vector ${\displaystyle P_i}$ along ${\displaystyle n}$, and equal to ${\displaystyle -d}$. So ${\displaystyle t=t\cdot 1=t\left(-\frac{n^T{P}_i}{d}\right)}$. And we have ${\displaystyle H_{ab}{P}_i=R{P}_i+t}$ where ${\displaystyle H_{ab}=R-\frac{t{n}^T}{d}}$.

This formula is only valid if camera b has no rotation and no translation. In the general case where ${\displaystyle R_a,R_b}$ and ${\displaystyle t_a,t_b}$ are the respective rotations and translations of camera a and b, ${\displaystyle R=R_a{R}_b^T}$ and the homography matrix ${\displaystyle H_{ab}}$ becomes

${\displaystyle H_{ab}=R_a{R}_b^T-\frac{(-R_a\ast R_b^T\ast t_b+t_a)n^T}{d}}$

where d is the distance of the camera b to the plane.