CODEWITHSUNDEEP
apache-sparkdatabricks
Luiz Viola
Luiz Viola

Reputation: 2436

How Databricks manages Spark Sessions in a colaborative cluster?

Consider we have several people working on a Databricks cluster in a colaborative way. Spark Context and Spark Session happens in the background.

Are they sharing the same Spark Session by default?

Upvotes: 1

Views: 1235

Answers (1)

Hubert Dudek
Hubert Dudek

Reputation: 1722

Spark Session Isolation is enabled by default. Users can disable session isolation by setting spark.databricks.session.share to true

You can set it in Spark config in advanced cluster option:

enter image description here

Upvotes: 2

Related Questions