Clustering Analysis: Comparing K-Means, GMM, and DBSCAN

A comparative study of three unsupervised clustering algorithms applied to a synthetic 2D dataset. The project covers exploratory data analysis, outlier detection, hyperparameter selection for each algorithm, and a final evaluation that weighs quantitative metrics against visual interpretation of cluster structure.

Overview

Given an unlabeled dataset with two continuous features, the goal is to identify the underlying cluster structure and compare how three different clustering paradigms handle it:

• K-Means — centroid-based, assumes spherical, equally-sized clusters
• Gaussian Mixture Model (GMM) — distribution-based, allows elliptical clusters of varying shape and orientation
• DBSCAN — density-based, identifies clusters of arbitrary shape and flags noise points directly

Each method is tuned independently using standard model-selection criteria, then all three are evaluated side by side using Silhouette Score and Davies-Bouldin Index.

Key Findings

• Exploratory density plots suggested 3–4 clusters, with an ambiguous elongated region that could plausibly be one or two clusters.
• K-Means selected k=3 (via Silhouette Score and Davies-Bouldin Index agreement), consolidating the ambiguous region into a single cluster.
• GMM selected k=4 (via BIC/AIC agreement), splitting that same region into two distinct elliptical components — a better visual match to the data's true structure.
• DBSCAN found 5 clusters but flagged 8.8% of points as noise, including two small clusters that look more like fragments than genuine groupings.
• Isolation Forest identified 2 clear outliers, both visually isolated from any cluster.
• On evaluation metrics, K-Means scored highest on Silhouette while DBSCAN scored best on Davies-Bouldin (the latter inflated by excluding 8.8% of the data as noise before scoring) — but GMM produced the most visually faithful clustering, despite ranking behind both on pure metrics.

This last point is the project's main takeaway: clustering metrics like Silhouette and Davies-Bouldin reward compact, well-separated clusters — properties K-Means directly optimizes for. That can bias evaluation toward K-Means even when another method (here, GMM) better reflects the data's actual shape. Metrics are a useful check, not a substitute for looking at the data.

Methods

1. Exploratory analysis — scatter and KDE density plots to visually estimate cluster count
2. Preprocessing — feature standardization with StandardScaler
3. Outlier detection — Isolation Forest with a data-driven anomaly score threshold
4. K-Means — swept k = 2–20, selected via Silhouette Score and Davies-Bouldin Index, cross-checked against the elbow method
5. Gaussian Mixture Model — swept k = 1–20, selected via BIC/AIC minimization
6. DBSCAN — min_samples set from feature count; eps selected via k-distance elbow detection
7. Evaluation — Silhouette Score and Davies-Bouldin Index computed for all three models, compared against visual cluster quality

Results Summary

Method	Clusters Found	Silhouette Score (↑)	Davies-Bouldin Score (↓)
K-Means	3	0.57186	0.60515
GMM	4	0.48983	0.78507
DBSCAN	5 (+ 8.8% noise)	0.36515	0.53838

Repository Structure

clustering-analysis-comparison/
├── data/
│   └── synthetic_2d_data.csv      # 500-row synthetic 2D dataset
├── notebooks/
│   └── clustering_analysis.ipynb  # full analysis notebook
├── requirements.txt
└── README.md

Running Locally

git clone https://github.com/SamuelMorgan777/clustering-analysis-comparison.git
cd clustering-analysis-comparison
python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate
pip install -r requirements.txt
jupyter notebook notebooks/clustering_analysis.ipynb

Tools

Python · pandas · NumPy · scikit-learn · SciPy · Matplotlib · Seaborn

Author

Samuel Morgan