CODEWITHSUNDEEP
scalaapache-sparkaccumulator
Puja Roy
Puja Roy

Reputation: 61

Accumulators not showing up in Spark UI

I have implemented Accumulators to see counts of events in my code. But I cannot see them in the Spark UI. I am using the following things :

val  count_Of_2317508_1 = sc.longAccumulator("count_Of_2317508_1")
 count_Of_2317508_1.add(1)

on some condition.

Is this enough for it to show up on the Spark UI?

Upvotes: 2

Views: 2810

Answers (1)

Anurag Sharma
Anurag Sharma

Reputation: 2605

You can create accumulators with or without a name, but only named accumulators are displayed in web UI (under Stages tab for a given stage).

Since you have already created named accumulator it should come.

val counter = sc.longAccumulator("counter")

enter image description here

Upvotes: 5

Related Questions