CODEWITHSUNDEEP
apache-sparkhadoop-yarnkerberos
tribbloid
tribbloid

Reputation: 3848

Why is Spark on YARN giving me 'User not found' not error?

When I submit spark-shell on a Kerberised YARN cluster. The YARN container quit immediately, Diagnositc in YARN application history shows that:

Application application_1515782018863_0007 failed 2 times due to AM Container for appattempt_1515782018863_0007_000002 exited with exitCode: -1000
For more detailed output, check application tracking page:http://ip-172-31-11-83.us-west-2.compute.internal:8088/cluster/app/application_1515782018863_0007Then, click on links to logs of each attempt.
Diagnostics: Application application_1515782018863_0007 initialization failed (exitCode=255) with output: **User datapass not found**
Failing this attempt. Failing the application.

The problem is that the user 'datapass' exist and is well-configured in the cluster:

kadmin.local:  listprincs
...
datapass/[email protected]
datapass/[email protected]
[email protected]
[email protected]
...

$ hadoop fs -ls /user
Found 9 items
drwxr-xr-x   - datapass datapass          0 2018-01-13 02:11 /user/datapass
drwxrwxrwx   - hadoop   hadoop            0 2018-01-12 18:50 /user/hadoop
drwxr-xr-x   - mapred   mapred            0 2018-01-12 18:33 /user/history
drwxrwxrwx   - hdfs     hadoop            0 2018-01-12 18:33 /user/hive
drwxrwxrwx   - hue      hue               0 2018-01-12 18:33 /user/hue
drwxrwxrwx   - livy     livy              0 2018-01-12 18:33 /user/livy
drwxrwxrwx   - oozie    oozie             0 2018-01-12 18:38 /user/oozie
drwxrwxrwx   - root     hadoop            0 2018-01-12 18:33 /user/root
drwxrwxrwx   - spark    spark             0 2018-01-12 18:33 /user/spark

So why YARN is giving me this false information?

Upvotes: 0

Views: 5831

Answers (1)

tribbloid
tribbloid

Reputation: 3848

Found the reason:

The user 'datapass' has to be created on the operating systems of all hadoop nodes, otherwise this weird error will be triggered.

Upvotes: 2

Related Questions