CODEWITHSUNDEEP
scalatwitterhadoopnoclassdeffounderrorscalding
neham
neham

Reputation: 341

Scalding Sample WordCount local mode

I am trying to run Scalding sample word count example. I have followed this github link for steps:- https://github.com/twitter/scalding/wiki/Getting-Started

But I am getting ClassNotFoundException. Below is my StackTrace:-

[cloudera@localhost scalding-develop]$ **sudo scripts/scald.rb --local WordCount --input       input.txt --output ./someOutputFile.tsv**
can not find /root/.sbt/boot/scala-2.9.3/lib/scala-library.jar appending SBT_VERSION [0.12.0] to SBT_HOME
scripts/scald.rb:139: warning: already initialized constant SBT_HOME
scripts/scald.rb:140: warning: already initialized constant SCALA_LIB_DIR
Exception in thread "main" java.lang.Throwable: If you know what exactly caused this error, please consider contributing to GitHub via following link.
https://github.com/twitter/scalding/wiki/Common-Exceptions-and-possible-reasons#javalangclassnotfoundexception
at com.twitter.scalding.Tool$.main(Tool.scala:146)
at com.twitter.scalding.Tool.main(Tool.scala)
Caused by: java.lang.ClassNotFoundException: WordCount
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:188)
at com.twitter.scalding.Job$.apply(Job.scala:39)
at com.twitter.scalding.Tool.getJob(Tool.scala:49)
at com.twitter.scalding.Tool.run(Tool.scala:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at com.twitter.scalding.Tool$.main(Tool.scala:132)
... 1 more

Please let me know where exactly is the issue?

Thanks.

Upvotes: 2

Views: 1336

Answers (3)

morazow
morazow

Reputation: 261

I suggest not using scald.rb. Instead use sbt, check this github repo, https://github.com/deanwampler/activator-scalding. It includes all the necessary basic setup.

Upvotes: 1

Antonios Chalkiopoulos
Antonios Chalkiopoulos

Reputation: 868

Check - Running your Scalding jobs in Eclipse article at http://hokiesuns.blogspot.co.uk/2012/07/running-your-scalding-jobs-in-eclipse.html

Using Maven for Scalding should be your first step into this new brave world. After a few months give sbt a try and see if you like it

Don't go directly to sbt as its usage is not that trivial

Upvotes: 1

debo
debo

Reputation: 11

Check if your JDK's bin directory is there in your PATH. I had a similar issue and I had use update-alternatives to install java but had not included /usr/lib/jvm/jdk-1.7.0/bin in my PATH.

Upvotes: 1

Related Questions