CODEWITHSUNDEEP
apache-sparkdatabricks
Carltonp
Carltonp

Reputation: 1344

Unable to run spark.eventLog.enabled true and spark.eventLog.dir on Databricks

When I attempt to run the the either line of code on my Databricks notebook I get the following error:

SyntaxError: invalid syntax at the keyword true

spark.eventLog.enabled true

and SyntaxError: invalid syntax at the keyword dbfs with the following command

spark.eventLog.dir dbfs:/databricks/unravel/eventLogs/

Upvotes: 1

Views: 918

Answers (1)

CHEEKATLAPRADEEP
CHEEKATLAPRADEEP

Reputation: 12768

You cannot configure the spark.eventLog using Databricks notebooks. I would suggest you to follow the below steps.

Steps to configure the Spark Event Logs:

Step1: Go to Clusters => Select a clusters

Step2: Click on Edit button

Step3: Under Advanced Options click Spark then Spark Config and enter the following:

spark.eventLog.enabled true
spark.eventLog.dir dbfs:/databricks/unravel/eventLogs/

Step4: Confirm and restart

enter image description here

enter image description here

Reference: Deploying Unravel for Azure Databricks from Azure Marketplace

Upvotes: 4

Related Questions