CODEWITHSUNDEEP
hadoop
Utumbu
Utumbu

Reputation: 442

Failed to read data from a hadoop URL

I am following Tom White's 'Hadoop - The Definitive Guide'. When I try to use the Java Interface to read data from a hadoop URL I get the following error message:

hadoop@ubuntu:/usr/local/hadoop$ hadoop URLCat hdfs://master/hdfs/data/SampleText.txt
12/11/21 13:46:32 INFO ipc.Client: Retrying connect to server: master/192.168.9.55:8020. Already tried 0 time(s).
12/11/21 13:46:33 INFO ipc.Client: Retrying connect to server: master/192.168.9.55:8020. Already tried 1 time(s).
12/11/21 13:46:34 INFO ipc.Client: Retrying connect to server: master/192.168.9.55:8020. Already tried 2 time(s).
12/11/21 13:46:35 INFO ipc.Client: Retrying connect to server: master/192.168.9.55:8020. Already tried 3 time(s).
12/11/21 13:46:36 INFO ipc.Client: Retrying connect to server: master/192.168.9.55:8020. Already tried 4 time(s).
12/11/21 13:46:37 INFO ipc.Client: Retrying connect to server: master/192.168.9.55:8020. Already tried 5 time(s).

The Contents of the URLCat file are as follows:

import java.net.URL;
import java.io.InputStream;
import org.apache.hadoop.io.IOUtils;
import org.apache.hadoop.fs.FsUrlStreamHandlerFactory;

public class URLCat {
static {
    URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory());
}

public static void main(String[] args) throws Exception {
    InputStream in = null;
    try {
        in = new URL(args[0]).openStream();
        IOUtils.copyBytes(in, System.out, 4096, false);
    } finally {
        IOUtils.closeStream(in);
    }
}
}

The /etc/hosts file contents are:

127.0.0.1   localhost
127.0.1.1   ubuntu.ubuntu-domain    ubuntu

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

# /ect/hosts Master and slaves
192.168.9.55    master
192.168.9.56    slave1
192.168.9.57    slave2
192.168.9.58    slave3

Upvotes: 0

Views: 2081

Answers (1)

Lorand Bendig
Lorand Bendig

Reputation: 10650

First I'd check whether the Hadoop daemons are running. A convenient tool is jps . Make sure that (at least) the namenode and the datanodes are running.

If you still can't connect, check whether the url is correct. As you provided hdfs://master/ (without any port number) Hadoop assumes that your namenode listens on port 8020 (default). This is what you see in the logs.

For a quick lookup in core-site.xml (fs.default.name) you can check whether you have a custom port defined for the filesystem URI (in this case 54310).

Upvotes: 1

Related Questions