Gfyuki

I've found two different approaches online when using the Elbow Method to determine the optimal number of clusters for K-Means.

One approach is to use the following code:

distortions_2.append(sum(np.min(cdist(data,

                                      kmeanModel.cluster_centers_,

                                      'euclidean'),

                                axis = 1)) / data.shape[0])

Another is to use inertia_ from sklearn.cluster.KMeans:

distortions_3.append(kmeanModel.inertia_)

When I plot the results (using the same random states) both give different results but I'm not sure what the differences are, can anyone help?

Edit: If I replace the normalisation factor / data.shape[0] with squared **2 as suggested below, then I still don't get the same as for the inertia plot:

distortions_2.append(sum(np.min(cdist(data, 

                                      kmeanModel.cluster_centers_, 

                                      'euclidean'), 

                                axis = 1)) ** 2)

Using squared just makes the plot a little smoother, but definitely not the same as using intertia_, I'm just not quite sure how inertia_ is calculated and what it means.

You will notice that the inertia is the sum of squared distance between each point and its nearest cluster center. Thus by removing the normalization term / data.shape[0] and instead replacing that by a squared **2 these two expressions will be equivalent

distortions_2.append(sum(np.min(cdist(data

                              , kmeanModel.cluster_centers_

                              , 'euclidean')

                              , axis = 1)) **2)

and

distortions_2.append(kmeanModel.inertia_)

For example using this artificial data source, where we generate 5 Gaussian distributions with 300 points in each.

params = [[[ 0,1],  [ 0,1]], 

          [[ 5,1],  [ 5,1]], 

          [[-2,5],  [ 2,5]],

          [[ 2,1],  [ 2,1]],

          [[-5,1],  [-5,1]]]



n = 300

dims = len(params[0])



data = 

y = 

for ix, i in enumerate(params):

    inst = np.random.randn(n, dims)

    for dim in range(dims):

        inst[:,dim] = params[ix][dim][0]+params[ix][dim][1]*inst[:,dim]

        label = ix + np.zeros(n)



    if len(data) == 0: data = inst

    else: data = np.append( data, inst, axis= 0)

    if len(y) == 0: y = label

    else: y = np.append(y, label)



num_clusters = len(params)

We will then apply KMeans using different number of clusters

k = [1,2,3,4,5,6,7,8,9,10]



inertias = 

dists = 



for i in k:

    kmeans = KMeans(i)

    kmeans.fit(data)

    inertias.append(kmeans.inertia_)

    dists.append(sum(np.min(spatial.distance.cdist(data, kmeans.cluster_centers_, 'euclidean'), axis=1)**2))



plt.plot(range(1, len(inertias)+1), inertias, label = 'Inertia')

plt.plot(range(1, len(dists)+1), dists, label = 'Distance')

plt.legend()

plt.show()