CODEWITHSUNDEEP
apache-spark
Rahul Sharma
Rahul Sharma

Reputation: 5834

How many RDDs in cache Spark

I have a spark application which cache RDDs at runtime based on datasets and perform operation.

For monitoring purpose I want to find out the number of RDDs in cache when application is running, does Spark provides any APIs to find out this details?

Upvotes: 0

Views: 184

Answers (1)

zero323
zero323

Reputation: 330393

It is possible to use Spark REST API which provides two endpoints:

  • /applications/[app-id]/storage/rdd - list of all stored RDDs.
  • /applications/[app-id]/storage/rdd/[rdd-id] - detail information for particular RDD.

Upvotes: 1

Related Questions