CODEWITHSUNDEEP
hadoopsqoopsqoop2
user236575
user236575

Reputation: 93

SQOOP Not able to import table

I am running below command on sqoop 

sqoop import --connect jdbc:mysql://localhost/hadoopguide --table widgets

my version of sqoop : Sqoop 1.4.4.2.0.6.1-101
Hadoop -- Hadoop 2.2.0.2.0.6.0-101

Both taken from hortonworks distribution. all the paths like HADOOP_HOME, HCAT_HOME, SQOOP_HOME are set properly. I am able to get list of databases, list of tables from mysql database by running list-database, list-tables commands in sqoop. Even able to get data from --query 'select * from widgets'; but when i use --table option getting below error. 

14/02/06 14:02:17 WARN mapred.LocalJobRunner: job_local177721176_0001
java.lang.Exception: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class widgets not found
        at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:403)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class widgets not found
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1720)
        at org.apache.sqoop.mapreduce.db.DBConfiguration.getInputClass(DBConfiguration.java:394)
        at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.createDBRecordReader(DataDrivenDBInputFormat.java:233)
        at org.apache.sqoop.mapreduce.db.DBInputFormat.createRecordReader(DBInputFormat.java:236)
        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.<init>(MapTask.java:491)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:734)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:339)
        at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:235)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.ClassNotFoundException: Class widgets not found
        at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1626)
        at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1718)
        ... 13 more

Upvotes: 3

Views: 5504

Answers (5)

tbone
tbone

Reputation: 15473

Another fix for ClassNotFoundException is to tell Hadoop to use the user classpath first (-Dmapreduce.job.user.classpath.first=true). This can be on command line or in Options file. The top of an import Options file would be:

#Options file for Sqoop import
import

-Dmapreduce.job.user.classpath.first=true

This fixed ClassNotFoundException for me when trying to import data as-avrodatafile

Upvotes: 0

Lehanshu Khatri
Lehanshu Khatri

Reputation: 1

For importing a specific table into hdfs, run:

sqoop import --connect jdbc:mysql://localhost/databasename --username root --password *** --table tablename --bindir /usr/lib/sqoop/lib/ --driver com.mysql.jdbc.Driver --target-dir /directory-name

Make sure that /usr/lib/sqoop/* and /usr/local/hadoop/* should be owned by the same user otherwise it will give error like "Permission denied".

PS: Make sure that you have installed mysql-java connector before you run the command. I installed hadoop version 2.7.3 and connector 5.0.8

Upvotes: 0

igaritano
igaritano

Reputation: 61

Use the --bindir option and point to your current working directory.

sqoop import --bindir ./ --connect jdbc:mysql://localhost/hadoopguide --table widgets

Upvotes: 6

user1639487
user1639487

Reputation: 61

Specify the --bindir where the compiled code and .jar file should be located.

Without these arguments, Sqoop would place the generated Java source file in your current working directory and the compiled .class file and .jar file in /tmp/sqoop-<username>/compile.

Upvotes: 6

user236575
user236575

Reputation: 93

The problem is resolved after i copied the .class file from /tmp/sqoop-hduser/compile/ to hdfs /home/hduser/ and also the current working directory from where i am running sqoop.

Upvotes: 1

Related Questions