CODEWITHSUNDEEP
apache-sparkprometheusstatsdapache-spark-3.0
Balu Reddy
Balu Reddy

Reputation: 1

apache spark metrics using StatsdSink

I am trying to monitor spark3 metrics using statdsink, prometheus and grafana but it looks like it is not sending driver, executor, stage and task level metrics. My cnfig file looks like below. I want to get all stage and task level metrics using prometheus, can anyone able to help me in this regard.

*.sink.statsd.class=org.apache.spark.metrics.sink.StatsdSink
*.sink.statsd.prefix=spark
*.sink.statsd.host=**********
*.sink.statsd.port=8125
master.source.jvm.class=org.apache.spark.metrics.source.JvmSource
worker.source.jvm.class=org.apache.spark.metrics.source.JvmSource
driver.source.jvm.class=org.apache.spark.metrics.source.JvmSource
executor.source.jvm.class=org.apache.spark.metrics.source.JvmSource

Upvotes: 0

Views: 134

Answers (0)

Related Questions