CODEWITHSUNDEEP
apache-spark
matanox
matanox

Reputation: 13706

Lost executor driver on localhost: Executor heartbeat timed out

I am debugging a spark application in local mode. Is it feasible to disable timeouts to avoid spark crashing in the middle of a debug session, without adverse effects?

Which timeout related parameters would you suggest changing for avoiding various timeouts without crashing spark altogether?

Currently I'm encountering this one, while debugging in IntelliJ:

Lost executor driver on localhost: Executor heartbeat timed out after 129006 ms

Upvotes: 0

Views: 2240

Answers (1)

Gelerion
Gelerion

Reputation: 1714

Add these two into the mix:

 val spark = SparkSession.builder()
  .config("spark.network.timeout", "10000001")
  .config("spark.executor.heartbeatInterval", "10000000")

Upvotes: 1

Related Questions