CODEWITHSUNDEEP
javahadoopapache-sparkhdfsapache-flink
Pa Rö
Pa Rö

Reputation: 459

HDFS Path for Spark Submit and Flink on YARN

i work with cloudera live vm, there i have a hadoop and spral standalone cluster. now i want submit my jobs with spark submit and flink run scripts. this works, too. but my apps can find the path to input and outputs files in the hdfs. i set the path like: hdfs://127.0.0.1:50010/user/cloudera/outputs the information from the port i get from this: enter image description here

how to set the path for the hdfs in java??

best regards, paul

Upvotes: 1

Views: 560

Answers (1)

Robert Metzger
Robert Metzger

Reputation: 4542

You don't have to set the path to the DataNode host. On the "Overview" page, you'll see the connection information for the NameNode, and that is also where you have to connect to.

Upvotes: 3

Related Questions