[scikit-learn/sklearn, pandas] Plot percent of variance explained for KMeans (Elbow Method)

eblow.py

import pandas as pd
import matplotlib.pyplot as plt
import seaborn
from sklearn.cluster import KMeans
import numpy as np
from scipy.spatial.distance import cdist, pdist

def elbow(df, n):
    kMeansVar = [KMeans(n_clusters=k).fit(df.values) for k in range(1, n)]
    centroids = [X.cluster_centers_ for X in kMeansVar]
    k_euclid = [cdist(df.values, cent) for cent in centroids]
    dist = [np.min(ke, axis=1) for ke in k_euclid]
    wcss = [sum(d**2) for d in dist]
    tss = sum(pdist(df.values)**2)/df.values.shape[0]
    bss = tss - wcss
    plt.plot(bss)
    plt.show()

I think you are missing from scipy.spatial.distance import cdist. And thanks for posting!

Missing pdist as well:
from scipy.spatial.distance import cdist, pdist

For an explanation: http://www.slideshare.net/SarahGuido/kmeans-clustering-with-scikitlearn

thanks! including

score function in from sklearn.cluster import KMeans gives the same graph pattern

Did you purposely spell this eblow instead of elbow?

@ericbf nope, didn't notice :). Thanks