CODEWITHSUNDEEP
apache-sparkmesos
IamMan
IamMan

Reputation: 442

Start Spark via Mesos

I am trying to start Spark via Mesos cluster, I have successfully run spark dispatcher via mesos master and register spark framework. But when i am trying to start any job (spark-submit or spark-shell) inicializing stop and happens nothing.

I0708 16:30:43.319941 21373 sched.cpp:157] Version: 0.22.1
I0708 16:30:43.331514 21375 sched.cpp:254] New master detected at master@host
I0708 16:30:43.333658 21375 sched.cpp:264] No credentials provided. Attempting to register without authentication

After that client freeze and that all

Upvotes: 3

Views: 733

Answers (1)

enrique-carbonell
enrique-carbonell

Reputation: 6452

I solved same problem with solution propose by @besil:

Step1: start the dispatcher using:

--master mesos://zk://<master_host>:2181/mesos

Step2: submit applications with:

spark-submit --deploy-mode cluster <other options> --master mesos://<dispatcher_ip>:7077 --class <class_name> <jar>

Hope this helps

Upvotes: 2

Related Questions