CODEWITHSUNDEEP
hadoop-yarnapache-zeppelin
Santosh Kumar
Santosh Kumar

Reputation: 791

Is it possible to run zeppelin with spark yarn-cluster

It runs well with spark yarn-client but what about yarn-cluster? If it's possible then please share how we can do that.

Upvotes: 5

Views: 2260

Answers (2)

Tagar
Tagar

Reputation: 14891

Yes, it is possible in current master version of Zeppelin.

Check out https://medium.com/@zjffdu/zeppelin-0-8-0-new-features-ea53e8810235

Yarn Cluster Mode for Spark Interpreter

Before 0.8.0, Zeppelin only supported yarn client mode for Spark Interpreter which means the driver would run in the same host of Zeppelin Server. The incur high memory pressure of the Zeppelin Server host especially when you run Spark Interpreter in isolated mode.

https://issues.apache.org/jira/browse/ZEPPELIN-2898 (merged to master end of Sep/2017).

Update: Notice 0.8.0 is released which has this feature in.

Upvotes: 3

Eran Witkon
Eran Witkon

Reputation: 4092

Currently zeppelin does not support yarn cluster, only yarn-client. What would be the usecase for this?

Upvotes: 0

Related Questions