CODEWITHSUNDEEP
hadoopmatrixmapreduce
thd
thd

Reputation: 2440

Read a specific block of rows from a sparse matrix in hadoop

I have a matrix store as follows

rowid, columnid, value

I want to read only a chunk of rows and send it to a mapper. For example, rows with id= 1,2,3,4 to a mapper, 5,6,7,8 to another one, ... Is it possible to do so ?

Upvotes: 0

Views: 80

Answers (1)

Joe K
Joe K

Reputation: 18434

No, this is probably not possible unless you jump through some serious hoops. This is what the reduce phase is for! Make the rowid your reduce key (or some calculation based on it... like (rowid-1)/4) and those chunks will be sent to the same reducer.

Upvotes: 1

Related Questions