CODEWITHSUNDEEP
db2sqoophadoop2data-ingestion
Vijay
Vijay

Reputation: 123

Issues with sqoop import from db2 to hdfs

I am trying to import data from DB2 to hdfs using sqoop. Since I do not have admin permissions to copy db2jcc4.jar to /var/lib/sqoop/ directory, I copied the jar to location where I am executing and specified that jar with --libjars option.

    sqoop import \
   --libjars db2jcc4.jar \
   --driver com.ibm.db2.jcc.DB2Driver \
   --connect 'jdbc:db2://<hostname>:50000/<dbname>' \
   --username <username> \
   --password <password> \
   --table <tablename> \
   --target-dir /hdfs path

I am getting the error as below:

   20/04/15 15:16:59 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.ibm.db2.jcc.DB2Driver
java.lang.RuntimeException: Could not load db driver class: com.ibm.db2.jcc.DB2Driver
        at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:875)
        at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:763)
        at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:786)
        at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:289)
        at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:260)
        at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:246)
        at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:327)
        at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1858)
        at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1657)
        at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)
        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:494)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:621)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:252)

Can anyone help me resolve the issue

Upvotes: 0

Views: 1046

Answers (1)

Fred Sobotka
Fred Sobotka

Reputation: 5332

Connecting to Db2 requires not just db2jcc4.jar but also its corresponding license JAR, typically db2jcc_license_cisuz.jar or db2jcc_license_cu.jar. The filename of the license JAR to use depends on the operating system of the Db2 server(s) you'll be accessing:

  • db2jcc_license_cu.jar can access Linux, UNIX, and Windows Db2 servers
  • db2jcc_license_cisuz.jar can access all Db2 server platforms, including z/OS and IBM i

If possible, ask your DBA to provide you with the right release level of db2jcc4.jar and a license JAR that's appropriate for your Db2 environment. Otherwise, you can download the IBM Data Server Driver for JDBC and SQLJ for free from IBM at https://www.ibm.com/support/pages/download-db2-fix-packs-version-db2-linux-unix-and-windows. As far as the license JAR is concerned, the JCC drivers available on that website may only include db2jcc_license_cu.jar and not db2jcc_license_cisuz.jar.

Both db2jcc JARs (driver and license) need to be listed in whatever classpath variable is used by your program. For sqoop, the environment variable is HADOOP_CLASSPATH, and you may also need to specify the --libjars option when running sqoop.

export HADOOP_CLASSPATH="/tmp/jdbcjars/db2jcc4.jar:/tmp/jdbcjars/db2jcc_license_cisuz.jar"
sqoop import --libjars /tmp/jdbcjars/db2jcc4.jar,/tmp/jdbcjars/db2jcc_license_cisuz.jar ...

Upvotes: 1

Related Questions