CODEWITHSUNDEEP
pandastensorflow
Benny Müller
Benny Müller

Reputation: 175

Transform sparse matrix to tensor

I have a pandas dataframe Object, containing a column with a bag of words representation of text stored as 29881x23947 sparse matrix of type ''. The column was processed by using sklearn and the fit_transform() function.

I now want to tranform this Colum into a 2D Tensor with the convert_to_tensor() function.

x_train_tensor = tf.convert_to_tensor(x_train)

Getting the error Message:

TypeError: Expected binary or unicode string

Which Format is required to Transform my Matrix into a Tensor?

Edit: printing out the type of the column it says: 

<class 'scipy.sparse.csr.csr_matrix'>

Example of the dataframe as output:

0          (0, 6276)\t1\n  (0, 8209)\t1\n  (0, 14299)\t...
1          (0, 6276)\t1\n  (0, 8209)\t1\n  (0, 14299)\t...
2          (0, 6276)\t1\n  (0, 8209)\t1\n  (0, 14299)\t...
3          (0, 6276)\t1\n  (0, 8209)\t1\n  (0, 14299)\t...

Upvotes: 5

Views: 4522

Answers (1)

Vijay Mariappan
Vijay Mariappan

Reputation: 17191

Here is an example of converting a sparse scipy matrix to tensorflow dense format.

Input sparse scipy matrix

A = np.array([[1,2,0],[0,0,3],[4,0,0]])
sA = sparse.csr_matrix(A)

print (sA)
# (0, 0)    1
# (0, 1)    2
# (1, 2)    3
# (2, 0)    4

idx, idy, val = sparse.find(sA)

print(idx, idy, val)
#[0 2 0 1] [0 0 1 2] [1 4 2 3]

To tensorflow

#merge idx and idy array to convert to [idx, idy] matrix    
full_indices = tf.stack([idx, idy], axis=1)

#Output matrix size
depth_x = 3
depth_y = 3

# sparse to dense matrix 
dense = tf.sparse_to_dense(full_indices,tf.constant([depth_x,depth_y]), val, validate_indices=False)

with tf.Session() as sess:
   print(sess.run(dense))
#[[1 2 0]
# [0 0 3]
# [4 0 0]]

Upvotes: 1

Related Questions