CODEWITHSUNDEEP
spring-xd
jitendra singh
jitendra singh

Reputation: 41

Getting error winutils.exe while running a stream in spirngxd with Hortonwork

I am using springXD 1.3.1 on my local pc with Windows7 enterprise version and configured Hortonworks on Microsoft Azure cloud. On Hortonworks at Azure, I created a dir xd and provided the required rights as mentioned in springxd doc. Then i made following entries in config/server.yml file:

spring:
  profiles: singlenode
  hadoop:
   fsUri: hdfs://13.92.199.104:8020
   resourceManagerHost: 13.92.199.104
   resourceManagerPort: 8050

Further, I made entry in config/hadoop.properties file as

fs.default.name=hdfs://13.92.199.104:8020

Then, i started springxd by command like xd-singlenode.bat and then started shell by xd-shell.bat

now on shell console i run command like 

hadoop config fs --namenode hdfs://13.92.199.104:8020

Till now, everything works fine. But, if now i run commands like hadoop fs ls /xd or if i try to deploy any simple stream like time | hdfs --fsUri=hdfs://13.92.199.104/ then getting following error like:

1.3.1.RELEASE | Admin Server Target: http://localhost:9393
Welcome to the Spring XD shell. For assistance hit TAB or type "help".
xd:>hadoop config fs --namenode hdfs://13.92.199.104:8020
xd:>hadoop fs ls /xd
2016-04-15T16:41:10+0530 ERROR Spring Shell o.a.h.u.Shell:374 - Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
        at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:356) [hadoop-common-2.7.1.jar:na]
        at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:371) [hadoop-common-2.7.1.jar:na]
        at org.apache.hadoop.util.Shell.<clinit>(Shell.java:364) [hadoop-common-2.7.1.jar:na]
        at org.apache.hadoop.fs.FsShellPermissions.<clinit>(FsShellPermissions.java:114) [hadoop-common-2.7.1.jar:na]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_77]
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_77]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_77]
        at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_77]
        at org.apache.hadoop.fs.shell.CommandFactory.registerCommands(CommandFactory.java:68) [hadoop-common-2.7.1.jar:na]
        at org.apache.hadoop.fs.shell.FsCommand.registerCommands(FsCommand.java:53) [hadoop-common-2.7.1.jar:na]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_77]
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_77]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_77]
        at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_77]
        at org.apache.hadoop.fs.shell.CommandFactory.registerCommands(CommandFactory.java:68) [hadoop-common-2.7.1.jar:na]
        at org.apache.hadoop.fs.FsShell.registerCommands(FsShell.java:101) [hadoop-common-2.7.1.jar:na]
        at org.apache.hadoop.fs.FsShell.init(FsShell.java:92) [hadoop-common-2.7.1.jar:na]
        at org.apache.hadoop.fs.FsShell.run(FsShell.java:274) [hadoop-common-2.7.1.jar:na]
        at org.springframework.xd.shell.hadoop.FsShellCommands.run(FsShellCommands.java:418) [spring-xd-shell-1.3.1.RELEASE.jar:1.3.1.RELEASE]
        at org.springframework.xd.shell.hadoop.FsShellCommands.runCommand(FsShellCommands.java:413) [spring-xd-shell-1.3.1.RELEASE.jar:1.3.1.RELEASE]
        at org.springframework.xd.shell.hadoop.FsShellCommands.ls(FsShellCommands.java:116) [spring-xd-shell-1.3.1.RELEASE.jar:1.3.1.RELEASE]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_77]
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_77]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[na:1.8.0_77]
        at java.lang.reflect.Method.invoke(Unknown Source) ~[na:1.8.0_77]
        at org.springframework.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:216) [spring-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
        at org.springframework.shell.core.SimpleExecutionStrategy.invoke(SimpleExecutionStrategy.java:64) [spring-shell-1.1.0.RELEASE.jar:1.1.0.RELEASE]
        at org.springframework.shell.core.SimpleExecutionStrategy.execute(SimpleExecutionStrategy.java:48) [spring-shell-1.1.0.RELEASE.jar:1.1.0.RELEASE]
        at org.springframework.shell.core.AbstractShell.executeCommand(AbstractShell.java:127) [spring-shell-1.1.0.RELEASE.jar:1.1.0.RELEASE]
        at org.springframework.shell.core.JLineShell.promptLoop(JLineShell.java:533) [spring-shell-1.1.0.RELEASE.jar:1.1.0.RELEASE]
        at org.springframework.shell.core.JLineShell.run(JLineShell.java:179) [spring-shell-1.1.0.RELEASE.jar:1.1.0.RELEASE]
        at java.lang.Thread.run(Unknown Source) [na:1.8.0_77]
Found 3 items
drwxrwxrwx   - jitendra.kumar.singh hdfs          0 2016-04-15 15:42 /xd/asdsadasdsad
drwxrwxrwx   - jitendra.kumar.singh hdfs          0 2016-04-15 14:30 /xd/fsd
drwxrwxrwx   - jitendra.kumar.singh hdfs          0 2016-04-15 14:34 /xd/timeLogHdfs

Is it windows specific error. how can i resolve it. Please suggest some way out of this problem.

Upvotes: 0

Views: 476

Answers (1)

Thomas Risberg
Thomas Risberg

Reputation: 976

Try downloading winutils.exe into a Hadoop "home" directory and set a system property pointing to this directory. See https://github.com/spring-projects/spring-hadoop/wiki/Using-a-Windows-client-together-with-a-Linux-cluster for more details. Not sure this will solve your problem, but it's worth a try.

Upvotes: 0

Related Questions