CODEWITHSUNDEEP
pythonpysparkgoogle-cloud-platformgoogle-cloud-dataproc
vortex
vortex

Reputation: 99

Errors for block matrix multiplification in Spark

I have created a coordinate matrix cmat with 9 million rows and 85K columns. I would like to perform cmat.T * cmat operations. I first converted cmat to block matrix bmat:

bmat = cmat.toBlockMatrix(1000, 1000)

However, I got errors when performing multiply():

mtm = bmat.transpose.multiply(bmat)

Traceback (most recent call last): File "", line 1, in AttributeError: 'function' object has no attribute 'multiply'

The Spark version is 2.2.0, scale version is 2.11.8 on DataProc, Google cloud platform. Any suggestions on how to fix the error?

Upvotes: 0

Views: 324

Answers (1)

MaFF
MaFF

Reputation: 10096

The error is saying that the result of operation bmat.transpose is a function not a blockMatrix and therefore has no attribute multiply.

You're just missing ():

mtm = bmat.transpose().multiply(bmat)

Upvotes: 2

Related Questions