CODEWITHSUNDEEP
hadoophdfsapache-nifihortonworks-data-platformhortonworks-dataflow
Flexron
Flexron

Reputation: 31

Apache Nifi Failed to write to HDFS

I'm getting this following error when I try to put files on HDFS using the PutHDFS processor :

ERROR

I work with the HortonWorks sandbox HDP and HDF (docker containers on Windows). It seem to be an Nifi's problem, HDFS is ok, I can put/write/read on it.

I'm trying to fix this but no result, I have deleted the docker's HDP image and got a new one, but no result too.

Upvotes: 3

Views: 1922

Answers (1)

Bryan Bende
Bryan Bende

Reputation: 18630

Usually that error means that whatever is writing to HDFS was able to reach the namenode but wasn't able to reach the datanode. In your case it is most likely a networking config issue between the two containers.

You could test this by eliminating NiFi from the scenario and install the Hadoop client on the container where NiFi is running, and try to use the Hadoop command line to write a file to HDFS. I would expect you get the same error.

If so make sure that you can ping the datanode hostname from the HDF container, make sure correct port is open, and make sure the datanode is accepting connections from all hosts and not just from localhost.

Upvotes: 4

Related Questions