CODEWITHSUNDEEP
hive
Shrashti
Shrashti

Reputation: 150

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask. No such file or directory

Getting the below exception in Hive on running simple SELECT COUNT(*) FROM Table.

Job Submission failed with exception 'org.apache.hadoop.io.nativeio.NativeIOException(No such file or directory)' FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask. No such file or directory

No issues occurs on simple SELECT * FROM Table.
Please suggest where might be the problem. Hive execution engine is MR.

Full stack trace of error :

2017-07-18T07:18:52,744 ERROR [main]: exec.Task (:()) - Job Submission failed with exception 'org.apache.hadoop.io.nativeio.NativeIOException(No such file or directory)' ENOENT: No such file or directory at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmodImpl(Native Method) at org.apache.hadoop.io.nativeio.NativeIO$POSIX.chmod(NativeIO.java:230) at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:729) at org.apache.hadoop.fs.ChecksumFileSystem$1.apply(ChecksumFileSystem.java:505) at org.apache.hadoop.fs.ChecksumFileSystem$FsOperation.run(ChecksumFileSystem.java:486) at org.apache.hadoop.fs.ChecksumFileSystem.setPermission(ChecksumFileSystem.java:502) at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:602) at org.apache.hadoop.mapreduce.JobResourceUploader.uploadFiles(JobResourceUploader.java:94) at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:95) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:190) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287) at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:575) at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:570) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:570) at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:561) at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:433) at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:138) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1858) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1562) at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1313) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1072) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399) at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:776) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:714) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.run(RunJar.java:221) at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

2017-07-18T07:18:52,745 ERROR [main]: ql.Driver (:()) - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask. No such file or directory

Upvotes: 1

Views: 3212

Answers (1)

Zdenek Kouba
Zdenek Kouba

Reputation: 51

Try checking permissions on hadoop tmp dir. The path to this directory is configured for example in core-site.xml. The property name is hadoop.tmp.dir. I had a similar issue as the one you described and it was caused by the user, under which the MR job was being executed, not having write permission to that location.

Upvotes: 2

Related Questions