CODEWITHSUNDEEP
hadoopcassandradatastax-enterprise
Pradyumn
Pradyumn

Reputation: 430

DSE with Hadoop: Error in getting started

I am facing a problem in DSE with Hadoop.

Let me describe the setup, including steps in some details, for you to be able to help me.

I set up a three-node cluster of DSE, with the cluster name as 'training'. All three machines are running Ubuntu 14.04, 64-bit, 4 GB RAM.

DSE was installed using the GUI installer (sudo command). After the installation, cassandra.yaml file was modified for

rpc_address = 0.0.0.0

One-by-one the three nodes were started. A keyspace with replication_factor = 3 was created. Data inserted and accessed from any other node, successfully.

Then DSE installed on a fourth machine (let's call this machine as HadoopMachine), again, with the same configuration, using GUI installer (sudo).

/etc/default/dse modified as follows:

HADOOP_ENABLED = 1

Then, on this HadoopMachine, the following command is run:

sudo service dse start

So far so good.

Then from the installation directory:

bin/dse hadoop fs -mkdir /user/hadoop/wordcount

This fails. Gives a very long series of error messages, running into hundreds of lines, ending with:

    at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1368)
    at com.datastax.bdp.loader.SystemClassLoader.tryLoadClassInBackground(SystemClassLoader.java:163)
    at com.datastax.bdp.loader.SystemClassLoader.loadClass(SystemClassLoader.java:117)
    at com.datastax.bdp.loader.SystemClassLoader.loadClass(SystemClassLoader.java:81)
    at com.datastax.bdp.loader.SystemClassLoader.loadClass(SystemClassLoader.java:75)
    at java.util.ResourceBundle$RBClassLoader.loadClass(ResourceBundle.java:503)
    at java.util.ResourceBundle$Control.newBundle(ResourceBundle.java:2640)
    at java.util.ResourceBundle.loadBundle(ResourceBundle.java:1501)
    at java.util.ResourceBundle.findBundle(ResourceBundle.java:1465)
    at java.util.ResourceBundle.findBundle(ResourceBundle.java:1419)
    at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1361)
    at java.util.ResourceBundle.getBundle(ResourceBundle.java:890)
    at sun.util.resources.LocaleData$1.run(LocaleData.java:164)
    at sun.util.resources.LocaleData$1.run(LocaleData.java:160)
FATAL ERROR in native method: processing of -javaagent failed
bin/dse: line 192: 12714 Aborted                 (core dumped) "$HADOOP_BIN/hadoop" "$HADOOP_CMD" $HADOOP_CREDENTIALS "${@:2}"

I don't know what the problem is, and how to fix it.

Will appreciate any help. Thanks.

Upvotes: 0

Views: 611

Answers (1)

Pradyumn
Pradyumn

Reputation: 430

I managed to find the solution, after a lot of struggle. I had been guessing all this time, that the problem would be one mis-step somewhere that was not very obvious, at least to me, and that's how it turned out.

So for the benefit of anybody else who may face the same problem, what the problem was and what worked is as follows.

DSE documentation specifies that for DSE with integrated Hadoop you must have Oracle JRE 7. I, perhaps foolishly, assumed it would mean Oracle JRE 7 or higher. So I had JRE 8 on my machine, and never realized that that would be the issue. When I removed JRE 8 and installed JRE 7, and bingo, it worked.

I am amazed. Now I realize that since DSE uses Hadoop 1.0.4 (an ancient version), it works with JRE 7 only. JRE 8 must have come after Hadoop 1.0.4 and something in JRE 8 must be incompatible with JRE 7, I guess.

Upvotes: 2

Related Questions