user2602577
Reputation: 1
Apache Spark Start multiple SparkContext instances
I would like to know how to create multiple SparkContext instances in the same application. Right now, the following message pops up: org.apache.spark.SparkException
Is it possible to run more than one SparkContext instances in the same application?
Upvotes: 0
Views: 708
Answers (1)
banjara
Reputation: 3890
Yes, it is possible by using the configuration below:
spark.driver.allowMultipleContexts=true
Reference: Support multiple SparkContexts in the same JVM
Upvotes: 1
Related Questions
- How to start multiple spark workers on one machine in Spark 2.4?
- Execute multiple Spark jobs
- Why can not run multiple SparkContexts at once
- how can we share the same spark context
- Spark multiple contexts
- How to create multiple SparkContexts in a console
- Spark starting more executors than specified
- Can I call the SparkContext constructor twice?
- More than one spark context error
- How to run multiple spark jobs in parallel with same spark context?