Hard and Soft Clustering
Definitions
Hard clustering (standard K-Means): every data point is assigned to exactly one cluster. Output: a single cluster label per point.
Soft clustering: every data point has a probability distribution over all clusters. Output: membership probabilities per cluster per point, summing to 1 across all clusters for each point.
Output Formats
| Aspect | Hard Clustering | Soft Clustering |
|---|---|---|
| Assignment | 1 cluster per point | Probabilities over K clusters |
| Example output | Point 3 → Cluster B | Point 3: {A: 0.7, B: 0.2, C: 0.1} |
| Interpretability | Simple, clear | Richer, nuanced |
| Evaluation | Standard (purity, ARI) | Harder (how to score a distribution?) |
When to Use Each
Hard Clustering
Appropriate when data has clear, non-overlapping natural groupings: objects belong to one category and only one. Examples:
- Grouping images of handwritten digits 0-9 (each digit is uniquely one class).
- Separating consumer products into discrete categories.
- Partitioning a network into disjoint communities where boundaries are sharp.
Soft Clustering
Appropriate when data has overlapping or ambiguous groupings: points may genuinely belong to multiple groups simultaneously. Examples:
- Social circles (Facebook Circles): a person may belong to family, work colleagues, and football club simultaneously. Soft clustering captures multiple simultaneous memberships.
- Document topic modelling: a document about “AI in healthcare” spans both “technology” and “medicine” topics. Soft clustering assigns partial membership to both.
- Gene expression: a gene may be involved in multiple biological pathways.
Trade-offs
Hard clustering is simpler to compute, simpler to interpret, and simpler to evaluate. But it forces points into artificial boundaries: a point exactly halfway between two equally valid cluster centres must arbitrarily choose one.
Soft clustering captures nuance and overlap but is harder to evaluate (what is the “correct” probability distribution?), harder to interpret (what does 40% membership in cluster A actually mean?), and computationally more intensive.
MLRD Course Context
The course emphasises that soft clustering is useful “when items naturally belong to overlapping groups, such as individuals belonging to multiple different overlapping social circles (as explored in the Facebook Circles data task).” This is drawn from the lecture material on unsupervised methods.
Summary
| Property | Hard Clustering | Soft Clustering |
|---|---|---|
| Membership | Exactly 1 cluster | Probabilistic, all clusters |
| Output | Label per point | Vector of probabilities per point |
| Use case | Non-overlapping natural groups | Overlapping groups, ambiguous boundaries |
| Interpretability | High | Moderate |
| Evaluation ease | High | Low |
| Example | Digit classification | Facebook social circles, topic models |
Past paper questions: y2024p3q7e (school class grouping uses hard clustering)