Fundamental matrix (computer vision)

Projective reconstruction theorem

The fundamental matrix can be determined by a set of point correspondences. Additionally, these corresponding image points may be triangulated to world points with the help of camera matrices derived directly from this fundamental matrix. The scene composed of these world points is within a projective transformation of the true scene.

Proof

Say that the image point correspondence ${\displaystyle \mathbf{x}\leftrightarrow {\mathbf{x}}^{\prime }}$ derives from the world point ${\displaystyle \mathbf{\text{X}}}$ under the camera matrices ${\displaystyle \left(\mathbf{\text{P}},{\mathbf{\text{P}}}^{\prime }\right)}$ as

${\displaystyle \begin{array}{cc}\mathbf{x}& =\mathbf{\text{P}}\mathbf{\text{X}}\\ {\mathbf{x}}^{\prime }& ={\mathbf{\text{P}}}^{\prime }\mathbf{\text{X}}\end{array}}$

Say we transform space by a general homography matrix ${\displaystyle {\mathbf{\text{H}}}_{4\times 4}}$ such that ${\displaystyle {\mathbf{\text{X}}}_0=\mathbf{\text{H}}\mathbf{\text{X}}}$.

The cameras then transform as

${\displaystyle \begin{array}{cc}{\mathbf{\text{P}}}_0& =\mathbf{\text{P}}{\mathbf{\text{H}}}^{-1}\\ {\mathbf{\text{P}}}_0^{\prime }& ={\mathbf{\text{P}}}^{\prime }{\mathbf{\text{H}}}^{-1}\end{array}}$

${\displaystyle {\mathbf{\text{P}}}_0{\mathbf{\text{X}}}_0=\mathbf{\text{P}}{\mathbf{\text{H}}}^{-1}\mathbf{\text{H}}\mathbf{\text{X}}=\mathbf{\text{P}}\mathbf{\text{X}}=\mathbf{x}}$ and likewise with ${\displaystyle {\mathbf{\text{P}}}_0^{\prime }}$ still get us the same image points.

Derivation of the fundamental matrix using coplanarity condition

The fundamental matrix can also be derived using the coplanarity condition.

For satellite images

The fundamental matrix expresses the epipolar geometry in stereo images. The epipolar geometry in images taken with perspective cameras appears as straight lines. However, in satellite images, the image is formed during the sensor movement along its orbit (pushbroom sensor). Therefore, there are multiple projection centers for one image scene and the epipolar line is formed as an epipolar curve. However, in special conditions such as small image tiles, the satellite images could be rectified using the fundamental matrix.

Properties

The fundamental matrix is of rank 2. Its kernel defines the epipole.

Toolboxes

• fundest is a GPL C/C++ library for robust, non-linear (based on the Levenberg–Marquardt algorithm) fundamental matrix estimation from matched point pairs and various objective functions (Manolis Lourakis).
• Structure and Motion Toolkit in MATLAB (Philip H. S. Torr)
• Fundamental Matrix Estimation Toolbox (Joaquim Salvi)
• The Epipolar Geometry Toolbox (EGT)