CODEWITHSUNDEEP
pythonmachine-learningk-meansdoc2vec
user1365234
user1365234

Reputation: 335

changing cluster labels for kmeans model

I have fit a Kmeans model on document embeddings from a Doc2Vec model to cluster the embeddings and get a visualization as well as the most frequent terms per cluster. I have been able to do this fine and get the same visualization each time.

When I run the kmeans.fit_predict on the model it gives me a list of cluster labels according to the clusters I have specified of the same length as the number of document embeddings I have. The issue comes when running the model multiple times it gives a similar spread per cluster each time but the cluster labels will change after running it multiple times. For example,

  1. Run 1 - 0:100, 1:100, 2:10
  2. Run 2 - 0:99 , 1:101, 2:10
  3. Run 3 - 2:100, 0:100, 1:10
  4. Run 4 - 0:100, 1:100, 2:10

I tried saving the model and using the same model multiple times but encountered the same issue. This causes the most frequent terms per cluster and position of the cluster in the visualization to change, which changes the way it is interpreted. I was planning to use the labels as a classification method but doesn't this make that impossible? I'm not sure if its an issue with my code or if this is normal behavior if anyone can help it would be much appreciated.

df = pd.read_csv("data.csv")
d2v_model = Doc2Vec.load("d2vmodel")

clusters = 3
iterations = 100

kmeans_model = KMeans(n_clusters=clusters, init='k-means++', max_iter=iterations) 
X = kmeans_model.fit(d2v_model.docvecs.vectors_docs)
l = kmeans_model.fit_predict(d2v_model.docvecs.vectors_docs)
labels = kmeans_model.labels_.tolist()

pca = PCA(n_components=2).fit(d2v_model.docvecs.vectors_docs)
datapoint = pca.transform(d2v_model.docvecs.vectors_docs)

df["clusters"] = labels
cluster_list = []
cluster_colors = ["#FFFF00", "#008000", "#0000FF"] 
plt.figure
color = [cluster_colors[i] for i in labels]
plt.scatter(datapoint[:, 0], datapoint[:, 1], c=color)
centroids = kmeans_model.cluster_centers_
centroidpoint = pca.transform(centroids)
plt.scatter(centroidpoint[:, 0], centroidpoint[:, 1], marker="^", s=150, c="#000000")

plt.show()

for i in range(clusters):
    df_temp = df[df["clusters"]==i]
    cluster_words = Counter(" ".join(df_temp["Body"].str.lower()).split()).most_common(25)
    [cluster_list.append(x[0]) for x in cluster_words]
    cluster_list.clear()

Upvotes: 1

Views: 3763

Answers (3)

A Rob4
A Rob4

Reputation: 1478

I think the issue might be use of .fit_predict. Try just .predict see https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html

try:

l = kmeans_model.predict(d2v_model.docvecs.vectors_docs)

similar worked for me

Upvotes: 0

gojomo
gojomo

Reputation: 54173

Stabilizing the initialization randomization by specifying a random_state (per @qaiser's answer) may help – perhaps by ensuring similar-ish sets of doc-vectors, against same starting KMeans state, tends to find the 'same' clusters in the same named slots.

But there could be situations, where the doc-vectors have a different distribution, or where initialized state is (by bad luck) highly sensitive to doc-vector distribution, where even this repeated-initialization doesn't maintain coherent clusters.

You might want to also consider one or both of:

(1) initializing the KMeans clusters to match the prior run's centroids, to bias the later analysis towards creating compatibly named/centered clusters;

(2) after the second run finishes, rename the clusters according to which (of all possible 3! arbitrary naming permutations of 3 clusters) leaves the smallest possible total distances between each 'new' cluster of the same name to the 'prior' cluster of the same name.

Upvotes: 0

qaiser
qaiser

Reputation: 2868

for Kmeans, when you run fit for multiple time, every time centroid will be initialized randomly. To make it deterministic you can use random_state parameters. you can refer to the docs https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html

 kmeans_model = KMeans(n_clusters=clusters, init='k-means++', max_iter=iterations, random_state = 'int number need to given')

Upvotes: 1

Related Questions