Curve orientation

In the case of a plane simple closed curve (that is, a curve in the plane whose starting point is also the end point and which has no other self-intersections), the curve is said to be positively oriented or counterclockwise oriented, if one always has the curve interior to the left (and consequently, the curve exterior to the right), when traveling on it. Otherwise, that is if left and right are exchanged, the curve is negatively oriented or clockwise oriented. This definition relies on the fact that every simple closed curve admits a well-defined interior, which follows from the Jordan curve theorem.

The inner loop of a beltway road in a country where people drive on the right side of the road is an example of a negatively oriented (clockwise) curve. In trigonometry, the unit circle is traditionally oriented counterclockwise.

The concept of orientation of a curve is just a particular case of the notion of orientation of a manifold (that is, besides orientation of a curve one may also speak of orientation of a surface, hypersurface, etc.).

Orientation of a curve is associated with parametrization of its points by a real variable. A curve may have equivalent parametrizations when there is a continuous increasing monotonic function relating the parameter of one curve to the parameter of the other. When there is a decreasing continuous function relating the parameters, then the parametric representations are opposite and the orientation of the curve is reversed.

Orientation of a simple polygon

In two dimensions, given an ordered set of three or more connected vertices (points) (such as in connect-the-dots) which forms a simple polygon, the orientation of the resulting polygon is directly related to the sign of the angle at any vertex of the convex hull of the polygon, for example, of the angle ABC in the picture. In computations, the sign of the smaller angle formed by a pair of vectors is typically determined by the sign of the cross product of the vectors. The latter one may be calculated as the sign of the determinant of their orientation matrix. In the particular case when the two vectors are defined by two line segments with common endpoint, such as the sides BA and BC of the angle ABC in our example, the orientation matrix may be defined as follows:

${\displaystyle \mathbf{O}=\left[\begin{array}{ccc}1& x_A& y_A\\ 1& x_B& y_B\\ 1& x_C& y_C\end{array}\right].}$

A formula for its determinant may be obtained, e.g., using the method of cofactor expansion:

${\displaystyle \begin{array}{cc}det(O)& =1\left|\begin{array}{cc}{x}_B& y_B\\ x_C& y_C\end{array}\right|-1\left|\begin{array}{cc}{x}_A& y_A\\ x_C& y_C\end{array}\right|+1\left|\begin{array}{cc}{x}_A& y_A\\ x_B& y_B\end{array}\right|\\ & =x_B{y}_C-y_B{x}_C-x_A{y}_C+y_A{x}_C+x_A{y}_B-y_A{x}_B\\ & =(x_B{y}_C+x_A{y}_B+y_A{x}_C)-(y_A{x}_B+y_B{x}_C+x_A{y}_C).\end{array}}$

If the determinant is negative, then the polygon is oriented clockwise. If the determinant is positive, the polygon is oriented counterclockwise. The determinant is non-zero if points A, B, and C are non-collinear. In the above example, with points ordered A, B, C, etc., the determinant is negative, and therefore the polygon is clockwise.

Practical considerations

In practical applications, the following considerations are commonly taken into account.

One does not need to construct the convex hull of a polygon to find a suitable vertex. A common choice is the vertex of the polygon with the smallest X-coordinate. If there are several of them, the one with the smallest Y-coordinate is picked. It is guaranteed to be a vertex of the convex hull of the polygon. Alternatively, the vertex with the smallest Y-coordinate among the ones with the largest X-coordinates or the vertex with the smallest X-coordinate among the ones with the largest Y-coordinates (or any other of 8 "smallest, largest" X/Y combinations) will do as well. Once a vertex of the convex hull is chosen, one can then apply the formula using the previous and next vertices, even if those are not on the convex hull, as there can be no local concavity on this vertex.

If the orientation of a convex polygon is sought, then, of course, any vertex may be picked.

For numerical reasons, the following equivalent formula for the determinant is commonly used:

${\displaystyle det(O)=(x_B-x_A)(y_C-y_A)-(x_C-x_A)(y_B-y_A)}$

The latter formula has four multiplications less. What is more important in computer computations involved in most practical applications, such as computer graphics or CAD, the absolute values of the multipliers are usually smaller (e.g., when A, B, C are within the same quadrant), thus giving a smaller numerical error or, in the extreme cases, avoiding the arithmetic overflow.

When it is not known in advance that the sequence of points defines a simple polygon, the following things must be kept in mind.

For a self-intersecting polygon (complex polygon) (or for any self-intersecting curve) there is no natural notion of the "interior", hence the orientation is not defined. At the same time, in geometry and computer graphics there are a number of concepts to replace the notion of the "interior" for closed non-simple curves; see, e.g., "flood fill" and "winding number".

In "mild" cases of self-intersection, with degenerate vertices when three consecutive points are allowed be on the same straight line and form a zero-degree angle, the concept of "interior" still makes sense, but an extra care must be taken in selection of the tested angle. In the given example, imagine point A to lie on segment BC. In this situation the angle ABC and its determinant will be 0, hence useless. A solution is to test consecutive corners along the polygon (BCD, DEF,...) until a non-zero determinant is found (unless all points lie on the same straight line). (Notice that the points C, D, E are on the same line and form a 180-degree angle with zero determinant.)