CODEWITHSUNDEEP
scalaapache-spark
x89a10
x89a10

Reputation: 691

Scala Spark RDD current number of partitions

How do you determine the number of partitions of an arbitrary RDD in Scala?

I know PySpark RDD has getNumPartitions defined in its API, but I can't locate the equivalent on the Scala side.

Upvotes: 4

Views: 4731

Answers (2)

echo
echo

Reputation: 1291

At least in Spark 1.6.1 this works

rdd.getNumPartitions()

Upvotes: 5

Gábor Bakos
Gábor Bakos

Reputation: 9100

It should be rdd.partitions.length.

Upvotes: 4

Related Questions