Reputation: 893
I would like to perform some batch computation after the stop of the streaming context. (strContext.stop(true,true)
I think that the stop of the streaming context will cause the stop of the associated spark context in fact I'm receiving an errore about the spark listener bus. It seems that even the call to strContext.stop(false,true) will cause the stop of the execution of the spark context.
Can anyone provide an example of batch computation after a streaming ?
Upvotes: 0
Views: 258
Reputation: 354
def stop(stopSparkContext: Boolean, stopGracefully: Boolean): Unit = {
...
if (stopSparkContext) sc.stop()
}
From the above code, spark Context should not be stopped if you pass strContext.stop(false).
Are you stopping some where else in you code. Can you paste your code here?
Upvotes: 1