CODEWITHSUNDEEP
apache-sparkrdd
drk
drk

Reputation: 173

what countByKey in JavaPairRDD

I have a JavaPairRDD named 'pair' and want to count the number of times a key occurs (I think the JavaPairRDD is not like a HashMap and will have keys repeated, am I right ?). The countByKey function returns an Object, instead of a Long. So Doesn't it return the count ? Or what is the Object returned, and can we find the count from it ?

Upvotes: 0

Views: 1392

Answers (1)

Glennie Helles Sindholt
Glennie Helles Sindholt

Reputation: 13154

JavaPairRDD.countByKey() returns a Map and the values are in fact the counts. Java has a bit of trouble with type inference in Spark (it's much, much better in Scala!), so you need to explicitly cast the values from Object to Long.

Upvotes: 1

Related Questions