How algorithms build hierarchies: the logic of bottom-up and top-down clustering
In short: Data mining often requires organizing complex information into nested structures. Hierarchical clustering achieves this by either building from individual points upward or splitting a whole into parts downward. While these methods are common, their results depend entirely on how you define 'similarity'—a choice that can fundamentally reshape the final map.
Hierarchical clustering operates through two primary strategies: agglomerative and divisive. Agglomerative clustering is a "bottom-up" approach where every data point begins as its own individual cluster. The algorithm iteratively merges the two most similar clusters based on a distance metric until all points are combined or a stopping criterion is met. Conversely, divisive clustering is a "top-down" method that starts with all data points in one large cluster and recursively splits them into smaller subsets. While agglomerative methods are more common due to their simplicity and efficiency for small to medium datasets, divisive methods can be useful for identifying large, distinct clusters first.
The outcome of these processes is typically visualized as a dendrogram. Because these algorithms generally operate in a greedy manner, they do not guarantee an optimum solution except in specific cases like single-linkage distance or exhaustive searches. How dissimilarity among single observations is quantified depends on the 'distance metric', whereas the 'linkage criterion' dictates how that difference for groups of data points is derived. For instance, Euclidean distance is the standard for continuous numerical data, but Manhattan distance is often preferred when dealing with outliers because it is less sensitive to large deviations.
The interaction between these two choices significantly influences the shape and quality of the final clusters. Single-linkage clustering, which uses the minimum distance between any pair of points in two clusters, can detect elongated or non-globular structures. In contrast, complete-linkage tends to produce more spherical clusters. For high-dimensional text mining, Cosine distance is preferred over basic geometry as it ignores vector magnitude; conversely, Mahalanobis distance factors in feature correlations by using the covariance matrix's inverse.
Opens the Intellect+ feed with this card first. Swipe up for the next one.
Sources
- Summary by Intellect · source: Hierarchical clustering
- Also: Bioinformatics advances — An optimization framework for hierarchical clustering. · CC BY
To cite this card, link to https://intellectplus.co.uk/app/health-hierarchical-clustering.
How we source, check and correct cards: Editorial standards.
General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.
Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?
