CODEWITHSUNDEEP
apache-sparkconfigurationsnappydata
mike w
mike w

Reputation: 141

SnappyData - configuring streaming job spark settings

I can see how to configure a SparkConf when creating a streaming application (see here)

I assume that I can configure the SparkConf through the SnappyStreamingContext for a streaming job similar to a streaming application. Let's say I get a handle to the SparkConf in a streaming job and modify some settings. Do these settings only apply to this streaming job or is this a global configuration update for all jobs?

thanks!

Upvotes: 1

Views: 119

Answers (1)

Yogesh Mahajan
Yogesh Mahajan

Reputation: 241

Yes, you can configure the SparkConf through the SnappyStreamingContext for a streaming job, and it is same as spark streaming configuration. Since SparkConf is a global configuration it is applicable to all the jobs in a streaming application. I think Spark doesn't allow you to change SparkConf after starting your application.

Upvotes: 1

Related Questions