orestis
orestis

Reputation: 972

Spark broadcast a value that is known at compilation time

I am running a Spark job and I have an object Constants that contains all constant values I am using in my program.

As an example let us assume that we have the following object:

 object Constants{
   ...
   val aSet=Set[String]("foo","bar","etc",...}
   ...
 }

Is it worth broadcasting the aSet val to the nodes, or is it already available to them since it is part of the code and it it is known in compilation time?

Upvotes: 1

Views: 96

Answers (1)

Graham S
Graham S

Reputation: 1712

My understanding is that every worker will have access to the object. Some related discussion here.

Upvotes: 2

Related Questions