CODEWITHSUNDEEP
javaspringhadoopspring-boothadoop-yarn
Markiza
Markiza

Reputation: 444

Spring Boot YARN how to deploy to Hadoop

I'm trying to run Spring Boot YARN sample (https://spring.io/guides/gs/yarn-basic/ on Windows). But example uses local hadoop. In application.yml I changed fsUri and resourceManagerHost to point to my VM's host 192.168....

But it doesn't work. When I'm moving jars to VM and run - it works. But I want to run from my host machine against another node (in my case VM).

And there is a question: what is the right way to run my app against another, real Hadoop YARN, not against local?

I've searched, read, watched a lot of info, but there were no definitive explanation about deploying.

Upvotes: 1

Views: 1211

Answers (2)

Ramesh Maharjan
Ramesh Maharjan

Reputation: 41977

The tutorial link provided in the question is for single node cluster where Namenode, Resourcemanager, ApplicationManager, datanode, Nodemanager and all other YARN nodes run on the same local machine. The address localhost:8020 works in such case.

But as you mention that the intention is to make a cluster, you will have to make sure there is only one namenode and resourcemanager.

If you define localhost in the configuration files then every nodes in the cluster will act namenode and resourcemanager and hadoop will get confused about reporting and updates queries. Thus resulting to ambiguous situation.

The solution is to define only one namenode and resourcemanager and this can be done by specifying an IP or hostname of the namenode (master) in the configuration files.

Upvotes: 1

Markiza
Markiza

Reputation: 444

Just needed right configuration of Hadoop, in my case it fixed in this post.

Upvotes: 1

Related Questions