CODEWITHSUNDEEP
scikit-learncosine-similarity
HugoLasticot
HugoLasticot

Reputation: 213

Cosine similarity TSNE in sklearn.manifold

I have a small problem to perform TSNE on my dataset, using cosine similarity.

I have calculated the cosine similarity of all of my vectors, so I have a square matrix which contains my cosine similarity : 

A = [[  1    0.7   0.5   0.6  ]
     [  0.7   1    0.3   0.4  ]
     [  0.5  0.3    1    0.1  ]
     [  0.6  0.4   0.1    1   ]]

Then, I'm using TSNE like that : 

A = np.matrix([[1, 0.7,0.5,0.6],[0.7,1,0.3,0.4],[0.5,0.3,1,0.1],[0.6,0.4,0.1,1]])
model = manifold.TSNE(metric="precomputed")
Y = model.fit_transform(A)

But I'm not sure that to use precomputed metric keep the sense of my cosine similarity: 

#[documentation][1]
If metric is “precomputed”, X is assumed to be a distance matrix

But when I try to use cosine metric, I got an error : 

A = np.matrix([[1, 0.7,0.5,0.6],[0.7,1,0.3,0.4],[0.5,0.3,1,0.1],[0.6,0.4,0.1,1]])
model = manifold.TSNE(metric="cosine")
Y = model.fit_transform(A) 

raise ValueError("All distances should be positive, either "
ValueError: All distances should be positive, either the metric or 
precomputed distances given as X are not correct

So my question is, How is it possible to perform TSNE using cosine metric on an existent dataset (similarity matrix) ?

Upvotes: 11

Views: 10529

Answers (3)

mrgloom
mrgloom

Reputation: 21682

Can be done with sklearn pairwise_distances:

from sklearn.manifold import TSNE
from sklearn.metrics import pairwise_distances

distance_matrix = pairwise_distances(X, X, metric='cosine', n_jobs=-1)
model = TSNE(metric="precomputed")
Xpr = model.fit_transform(distance_matrix)

Values in distance_matrix will be in [0,2] range, because (1 - [-1,1]).

Upvotes: 3

o17t H1H' S'k
o17t H1H' S'k

Reputation: 2745

there is currently a bug. see here: https://github.com/scikit-learn/scikit-learn/issues/5772

however scikit's t-sne uses the squared euclidean distance which is proportional to the cosine distance, assuming your data is L2 normalized

Upvotes: 2

piman314
piman314

Reputation: 5355

I can answer the majority of your question, however I'm not quite sure why that error is popping up in your second example.

You have calculated the cosine similarity of each of your vectors, but scikit assumes a distance matrix for the input to TSNE. However this is a really simple transformation distance = 1 - similarity. So for your example

import numpy as np
from sklearn import manifold
A = np.matrix([[1, 0.7,0.5,0.6],[0.7,1,0.3,0.4],[0.5,0.3,1,0.1],[0.6,0.4,0.1,1]])
A = 1.-A
model = manifold.TSNE(metric="precomputed")
Y = model.fit_transform(A)

This should give you the transformation you want.

Upvotes: 7

Related Questions