Great-circle navigation

Course

The great circle path may be found using spherical trigonometry; this is the spherical version of the inverse geodetic problem. If a navigator begins at P₁ = (φ₁,λ₁) and plans to travel the great circle to a point at point P₂ = (φ₂,λ₂) (see Fig. 1, φ is the latitude, positive northward, and λ is the longitude, positive eastward), the initial and final courses α₁ and α₂ are given by formulas for solving a spherical triangle

${\displaystyle \begin{array}{cc}\tan {\alpha }_1& =\frac{\cos {\varphi }_2\sin {\lambda }_{12}}{\cos {\varphi }_1\sin {\varphi }_2-\sin {\varphi }_1\cos {\varphi }_2\cos {\lambda }_{12}},\\ \tan {\alpha }_2& =\frac{\cos {\varphi }_1\sin {\lambda }_{12}}{-\cos {\varphi }_2\sin {\varphi }_1+\sin {\varphi }_2\cos {\varphi }_1\cos {\lambda }_{12}},\end{array}}$

where λ₁₂ = λ₂ − λ₁ and the quadrants of α₁,α₂ are determined by the signs of the numerator and denominator in the tangent formulas (e.g., using the atan2 function). The central angle between the two points, σ₁₂, is given by

${\displaystyle \tan {\sigma }_{12}=\frac{\sqrt{(\cos {\varphi }_1\sin {\varphi }_2-\sin {\varphi }_1\cos {\varphi }_2\cos {\lambda }_{12}{)}^2+(\cos {\varphi }_2\sin {\lambda }_{12}{)}^2}}{\sin {\varphi }_1\sin {\varphi }_2+\cos {\varphi }_1\cos {\varphi }_2\cos {\lambda }_{12}}.}$

(The numerator of this formula contains the quantities that were used to determine tan α₁.) The distance along the great circle will then be s₁₂ = Rσ₁₂, where R is the assumed radius of the Earth and σ₁₂ is expressed in radians. Using the mean Earth radius, R = R₁ ≈ 6,371 km (3,959 mi) yields results for the distance s₁₂ which are within 1% of the geodesic length for the WGS84 ellipsoid; see Geodesics on an ellipsoid for details.

Relation to geocentric coordinate system

Detailed evaluation of the optimum direction is possible if the sea surface is approximated by a sphere surface. The standard computation places the ship at a geodetic latitude φ_s and geodetic longitude λ_s, where φ is considered positive if north of the equator, and where λ is considered positive if east of Greenwich. In the geocentric coordinate system centered at the center of the sphere, the Cartesian components are

${\displaystyle \mathbf{s}=R\left(\begin{array}{c}\cos {\phi }_s\cos {\lambda }_s\\ \cos {\phi }_s\sin {\lambda }_s\\ \sin {\phi }_s\end{array}\right)}$

and the target position is

${\displaystyle \mathbf{t}=R\left(\begin{array}{c}\cos {\phi }_t\cos {\lambda }_t\\ \cos {\phi }_t\sin {\lambda }_t\\ \sin {\phi }_t\end{array}\right).}$

The North Pole is at

${\displaystyle \mathbf{N}=R\left(\begin{array}{c}0\\ 0\\ 1\end{array}\right).}$

The minimum distance d is the distance along a great circle that runs through s and t. It is calculated in a plane that contains the sphere center and the great circle,

${\displaystyle d_{s,t}=R{\theta }_{s,t}}$

where θ is the angular distance of two points viewed from the center of the sphere, measured in radians. The cosine of the angle is calculated by the dot product of the two vectors

${\displaystyle \mathbf{s}\cdot \mathbf{t}=R^2\cos {\theta }_{s,t}=R^2(\sin {\phi }_s\sin {\phi }_t+\cos {\phi }_s\cos {\phi }_t\cos ({\lambda }_t-{\lambda }_s))}$

If the ship steers straight to the North Pole, the travel distance is

${\displaystyle d_{s,N}=R{\theta }_{s,N}=R(\pi /2-{\phi }_s)}$

If a ship starts at t and swims straight to the North Pole, the travel distance is

${\displaystyle d_{t,N}=R{\theta }_{t,n}=R(\pi /2-{\phi }_t)}$

Derivation

The cosine formula of spherical trigonometry yields for the angle p between the great circles through s that point to the North on one hand and to t on the other hand

${\displaystyle \cos {\theta }_{t,N}=\cos {\theta }_{s,t}\cos {\theta }_{s,N}+\sin {\theta }_{s,t}\sin {\theta }_{s,N}\cos p.}$

${\displaystyle \sin {\phi }_t=\cos {\theta }_{s,t}\sin {\phi }_s+\sin {\theta }_{s,t}\cos {\phi }_s\cos p.}$

The sine formula yields

${\displaystyle \frac{\sin p}{\sin {\theta }_{t,N}}=\frac{\sin ({\lambda }_t-{\lambda }_s)}{\sin {\theta }_{s,t}}.}$

Solving this for sin θ_s,t and insertion in the previous formula gives an expression for the tangent of the position angle,

${\displaystyle \sin {\phi }_t=\cos {\theta }_{s,t}\sin {\phi }_s+\frac{\sin ({\lambda }_t-{\lambda }_s)}{\sin p}\cos {\phi }_t\cos {\phi }_s\cos p;}$

${\displaystyle \tan p=\frac{\sin ({\lambda }_t-{\lambda }_s)\cos {\phi }_t\cos {\phi }_s}{\sin {\phi }_t-\cos {\theta }_{s,t}\sin {\phi }_s}.}$

Further details

Because the brief derivation gives an angle between 0 and π which does not reveal the sign (west or east of north ?), a more explicit derivation is desirable which yields separately the sine and the cosine of p such that use of the correct branch of the inverse tangent allows to produce an angle in the full range -π≤p≤π.

The computation starts from a construction of the great circle between s and t. It lies in the plane that contains the sphere center, s and t and is constructed rotating s by the angle θ_s,t around an axis ω. The axis is perpendicular to the plane of the great circle and computed by the normalized vector cross product of the two positions:

${\displaystyle \omega =\frac{1}{R^2\sin {\theta }_{s,t}}\mathbf{s}\times \mathbf{t}=\frac{1}{\sin {\theta }_{s,t}}\left(\begin{array}{c}\cos {\phi }_s\sin {\lambda }_s\sin {\phi }_t-\sin {\phi }_s\cos {\phi }_t\sin {\lambda }_t\\ \sin {\phi }_s\cos {\lambda }_t\cos {\phi }_t-\cos {\phi }_s\sin {\phi }_t\cos {\lambda }_s\\ \cos {\phi }_s\cos {\phi }_t\sin ({\lambda }_t-{\lambda }_s)\end{array}\right).}$

A right-handed tilted coordinate system with the center at the center of the sphere is given by the following three axes: the axis s, the axis

${\displaystyle {\mathbf{s}}_{\perp }=\omega \times \frac{1}{R}\mathbf{s}=\frac{1}{\sin {\theta }_{s,t}}\left(\begin{array}{c}\cos {\phi }_t\cos {\lambda }_t({\sin }^2{\phi }_s+{\cos }^2{\phi }_s{\sin }^2{\lambda }_s)-\cos {\lambda }_s(\sin {\phi }_s\cos {\phi }_s\sin {\phi }_t+{\cos }^2{\phi }_s\sin {\lambda }_s\cos {\phi }_t\sin {\lambda }_t)\\ \cos {\phi }_t\sin {\lambda }_t({\sin }^2{\phi }_s+{\cos }^2{\phi }_s{\cos }^2{\lambda }_s)-\sin {\lambda }_s(\sin {\phi }_s\cos {\phi }_s\sin {\phi }_t+{\cos }^2{\phi }_s\cos {\lambda }_s\cos {\phi }_t\cos {\lambda }_t)\\ \cos {\phi }_s[\cos {\phi }_s\sin {\phi }_t-\sin {\phi }_s\cos {\phi }_t\cos ({\lambda }_t-{\lambda }_s)]\end{array}\right)}$

and the axis ω. A position along the great circle is

${\displaystyle \mathbf{s}(\theta )=\cos \theta \mathbf{s}+\sin \theta {\mathbf{s}}_{\perp },0\le \theta \le 2\pi .}$

The compass direction is given by inserting the two vectors s and s_⊥ and computing the gradient of the vector with respect to θ at θ=0.

${\displaystyle \frac{\mathrm{\partial }}{\mathrm{\partial }\theta }{\mathbf{s}}_{\mid \theta =0}={\mathbf{s}}_{\perp }.}$

The angle p is given by splitting this direction along two orthogonal directions in the plane tangential to the sphere at the point s. The two directions are given by the partial derivatives of s with respect to φ and with respect to λ, normalized to unit length:

${\displaystyle {\mathbf{u}}_N=\left(\begin{array}{c}-\sin {\phi }_s\cos {\lambda }_s\\ -\sin {\phi }_s\sin {\lambda }_s\\ \cos {\phi }_s\end{array}\right);}$

${\displaystyle {\mathbf{u}}_E=\left(\begin{array}{c}-\sin {\lambda }_s\\ \cos {\lambda }_s\\ 0\end{array}\right);}$

${\displaystyle {\mathbf{u}}_N\cdot \mathbf{s}={\mathbf{u}}_E\cdot {\mathbf{u}}_N=0}$

u_N points north and u_E points east at the position s. The position angle p projects s_⊥ into these two directions,

${\displaystyle {\mathbf{s}}_{\perp }=\cos p{\mathbf{u}}_N+\sin p{\mathbf{u}}_E}$,

where the positive sign means the positive position angles are defined to be north over east. The values of the cosine and sine of p are computed by multiplying this equation on both sides with the two unit vectors,

${\displaystyle \cos p={\mathbf{s}}_{\perp }\cdot {\mathbf{u}}_N=\frac{1}{\sin {\theta }_{s,t}}[\cos {\phi }_s\sin {\phi }_t-\sin {\phi }_s\cos {\phi }_t\cos ({\lambda }_t-{\lambda }_s)];}$

${\displaystyle \sin p={\mathbf{s}}_{\perp }\cdot {\mathbf{u}}_E=\frac{1}{\sin {\theta }_{s,t}}[\cos {\phi }_t\sin ({\lambda }_t-{\lambda }_s)].}$

Instead of inserting the convoluted expression of s_⊥, the evaluation may employ that the triple product is invariant under a circular shift of the arguments:

${\displaystyle \cos p=(\omega \times \frac{1}{R}\mathbf{s})\cdot {\mathbf{u}}_N=\omega \cdot (\frac{1}{R}\mathbf{s}\times {\mathbf{u}}_N).}$

If atan2 is used to compute the value, one can reduce both expressions by division through cos φ_t and multiplication by sin θ_s,t, because these values are always positive and that operation does not change signs; then effectively

${\displaystyle \tan p=\frac{\sin ({\lambda }_t-{\lambda }_s)}{\cos {\phi }_s\tan {\phi }_t-\sin {\phi }_s\cos ({\lambda }_t-{\lambda }_s)}.}$