CODEWITHSUNDEEP
hadoopmkdir
user1467855
user1467855

Reputation: 3993

hadoop creates dir that cannot be found

I use the following hadoop command to create a directory

 hdfs dfs -mkdir /tmp/testing/morehere1

I get the following message:

 log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).

log4j:WARN Please initialize the log4j system properly.

Not understanding the error, I run the command again, which returns this message:

 log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory).
 log4j:WARN Please initialize the log4j system properly.
 mkdir: `/tmp/testing/morehere2': File exists

then when I try to go to the directory just created, it's not there.

cd /tmp/testing/morehere2
 -bash: cd: /tmp/testing/morehere2: No such file or directory

Any ideas what I am doing wrong?

Upvotes: 0

Views: 7715

Answers (3)

Manish Agrawal
Manish Agrawal

Reputation: 794

If your MR code were running fine previously and Now its showing this log4j error then restart all the hadoop daemons. It may solve your problem as it solves mine :)

Upvotes: 0

hari_sree
hari_sree

Reputation: 1538

 hdfs dfs -mkdir /tmp/testing/morehere1

This command created a directory in the hdfs . Dont worry about the log4j warning . The command created the directory successfully . That is why you got the error mkdir: /tmp/testing/morehere2': File exists the second time you tried the command .

The following command will not work , since the directory is not created in your local filesystem , but in hdfs .

cd /tmp/testing/morehere2

Use the command below to check the created directory in hdfs : 

hdfs dfs -ls /tmp/testing

You should be able to see the new directory there .

About the log4j warning : You can ignore that as it will not cause your hadoop commands to fail . But if you want to correct it , you can add a File appender to log4j.properties .

Upvotes: 5

Joe K
Joe K

Reputation: 18424

Remember that there's a difference between HDFS and your local file system. That first line that you posted creates a directory in HDFS, not on your local system. So you can't cd to it or ls it or anything directly; if you want to access it, you have to through hadoop. It's also very rare to be logging to HDFS as file appends have never been well-supported. I suspect that you actually want to be creating that directory locally, and that might be part of your problem.

Upvotes: 1

Related Questions