Silhouette (clustering)

The silhouette value is a measure of how similar an object is to its own cluster (cohesion) compared to other clusters (separation). The silhouette value ranges from −1 to +1, where a high value indicates that the object is well matched to its own cluster and poorly matched to neighboring clusters. If most objects have a high value, then the clustering configuration is appropriate. If many points have a low or negative value, then the clustering configuration may have too many or too few clusters. A clustering with an average silhouette width of over 0.7 is considered to be "strong", a value over 0.5 "reasonable", and over 0.25 "weak". However, with an increasing dimensionality of the data, it becomes difficult to achieve such high values because of the curse of dimensionality, as the distances become more similar. The silhouette score is specialized for measuring cluster quality when the clusters are convex-shaped, and may not perform well if the data clusters have irregular shapes or are of varying sizes. The silhouette value can be calculated with any distance metric, such as Euclidean distance or Manhattan distance.

Definition

Assume the data have been clustered via any technique, such as k-medoids or k-means, into ${\displaystyle k}$ clusters.

For data point ${\displaystyle i\in C_i}$ (data point ${\displaystyle i}$ in the cluster ${\displaystyle C_i}$), calculate ${\displaystyle a(i)}$, the average distance that ${\displaystyle i}$ is from all other points in that cluster:

${\displaystyle a(i)=\frac{1}{|C_i|-1}\sum _{j\in C_i,i\ne j}d(i,j)}$

where ${\displaystyle |C_i|}$ is the number of points belonging to cluster ${\displaystyle C_i}$, and ${\displaystyle d(i,j)}$ is the distance between data points ${\displaystyle i}$ and ${\displaystyle j}$ in the cluster ${\displaystyle C_i}$ (we divide by ${\displaystyle |C_i|-1}$ because the distance ${\displaystyle d(i,i)}$ is not included in the sum). ${\displaystyle a(i)}$ can be interpreted as a measure of how well ${\displaystyle i}$ is assigned to its cluster (the smaller the value, the better the assignment).

We then define the mean dissimilarity of point ${\displaystyle i}$ to some cluster ${\displaystyle C_j}$ as the mean of the distance from ${\displaystyle i}$ to all points in ${\displaystyle C_j}$ (where ${\displaystyle C_j\ne C_i}$).

For each data point ${\displaystyle i\in C_i}$, we now define ${\displaystyle b(i)}$ as the average distance between ${\displaystyle i}$ and the points in the closest cluster (hence: "min") that ${\displaystyle i}$ does not belong to:

${\displaystyle b(i)=\underset{j\ne i}{min}\frac{1}{|C_j|}\sum _{l\in C_j}d(i,l)}$

The cluster with the smallest mean dissimilarity is said to be the "neighboring cluster" of ${\displaystyle i}$ because it is the next best fit cluster for point ${\displaystyle i}$.

We now define a silhouette (value) of one data point ${\displaystyle i}$

${\displaystyle s(i)=\frac{b(i)-a(i)}{max\{a(i),b(i)\}}}$, if ${\displaystyle |C_i|>1}$

and

${\displaystyle s(i)=0}$, if ${\displaystyle |C_i|=1}$,

which can also be written as

${\displaystyle s(i)=\{\begin{array}{ll}1-\frac{a(i)}{b(i)},& \text{ if }a(i)<b(i)\\ 0,& \text{if }a(i)=b(i)\\ \frac{b(i)}{a(i)}-1,& \text{ if }a(i)>b(i)\end{array}}$

From the above definition, ${\displaystyle s(i)}$ is bounded to the interval ${\displaystyle [-1,1]}$, i.e. ${\displaystyle -1\le s(i)\le 1.}$

Note that ${\displaystyle a(i)}$ is not clearly defined for clusters with size = 1, in which case we set ${\displaystyle s(i)=0}$. This choice is arbitrary, but neutral in the sense that it is at the midpoint of the bounds, -1 and 1.

For ${\displaystyle s(i)}$ to be close to 1 we require ${\displaystyle a(i)\ll b(i)}$. As ${\displaystyle a(i)}$ is a measure of how dissimilar ${\displaystyle i}$ is to its own cluster, a small value means it is well matched. Furthermore, a large ${\displaystyle b(i)}$ implies that ${\displaystyle i}$ is badly matched to its neighbouring cluster. Thus an ${\displaystyle s(i)}$ close to 1 means that the data is appropriately clustered. If ${\displaystyle s(i)}$ is close to -1, then by the same logic we see that ${\displaystyle i}$ would be more appropriate if it was clustered in its neighbouring cluster. An ${\displaystyle s(i)}$ near zero means that the datum is on the border of two natural clusters.

The mean ${\displaystyle s(i)}$ over all points of a cluster is a measure of how tightly grouped all the points in the cluster are. Thus the mean ${\displaystyle s(i)}$ over all data of the entire dataset is a measure of how appropriately the data have been clustered. If there are too many or too few clusters, as may occur when a poor choice of ${\displaystyle k}$ is used in the clustering algorithm (e.g., k-means), some of the clusters will typically display much narrower silhouettes than the rest. Thus silhouette plots and means may be used to determine the natural number of clusters within a dataset. One can also increase the likelihood of the silhouette being maximized at the correct number of clusters by re-scaling the data using feature weights that are cluster specific.

Kaufman et al. introduced the term silhouette coefficient for the maximum value of the mean ${\displaystyle s(i)}$ over all data of the entire dataset, i.e.,

${\displaystyle SC=\underset{k}{max}\tilde{s}\left(k\right),}$

where ${\displaystyle \tilde{s}\left(k\right)}$ represents the mean ${\displaystyle s(i)}$ over all data of the entire dataset for a specific number of clusters ${\displaystyle k}$. The silhouette coefficient describes the best possible clustering possible for a given number of clusters, as measured by the highest average silhouette score for all points in the dataset.