CODEWITHSUNDEEP
teradata
priyanka
priyanka

Reputation: 325

teradata export from hive to teradata class not found

I am trying to export from hive table to teradata using TDCH connector, I am getting below errror:-

15/05/07 08:01:03 INFO tool.ConnectorExportTool: java.lang.NoClassDefFoundError: org/apache/hadoop/hive/metastore/api/MetaException
            at java.lang.Class.forName0(Native Method)
            at java.lang.Class.forName(Class.java:190)
            at com.teradata.connector.common.tool.ConnectorJobRunner.runJob(ConnectorJobRunner.java:81)
            at com.teradata.connector.common.tool.ConnectorExportTool.run(ConnectorExportTool.java:61)
            at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
            at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
            at com.teradata.hadoop.tool.TeradataExportTool.main(TeradataExportTool.java:24)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:606)
            at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
            at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
    Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.metastore.api.MetaException
            at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
            at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
            at java.security.AccessController.doPrivileged(Native Method)
            at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
            at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
            ... 13 more

I understand from error that hive-metastore jar is missing. But it is already there in hive/lib folder

 hive-metastore.jar -> hive-metastore-0.9.0.jar

Is already present in the path: /usr/hdp/2.2.4.2-2/hive/lib

Upvotes: 1

Views: 2352

Answers (3)

Heapify
Heapify

Reputation: 2901

Sometimes you may have all the jar files in your "lib" folder, but for some reason Oozie may not be reading it simply because you have a typo in your job.properties or coordinator.properties. Double check your properties file.

The "job.properties" file should read like the following - 

oozie.use.system.libpath=true
oozie.wf.application.path=${nameNode}/apps/myapp/workflow/

The "coordinator.properties" file should read like the following - 

oozie.use.system.libpath=true
oozie.coord.application.path=${nameNode}/apps/myapp/workflow/

and your "lib" folder with jars should be at -

/apps/myapp/workflow/lib

Upvotes: 0

user3452265
user3452265

Reputation: 36

You have to define environment variables LIB_JARS and HADOOP_CLASSPATH. You can use the environment variable LIB_JARS with the parameter -libjars

You can find an example under https://developer.teradata.com/sites/all/files/Teradata%20Connector%20for%20Hadoop%20Tutorial%20v1%200%20final.pdf.

According to the README, following jars are necessairy:

    Hive Job(version 0.11.0 as example):
         a) hive-metastore-0.11.0.jar
         b) hive-exec-0.11.0.jar
         c) hive-cli-0.11.0.jar
         d) libthrift-0.9.0.jar
         e) libfb303-0.9.0.jar
         f) jdo2-api-2.3-ec.jar
         g) slf4j-api-1.6.1.jar
         h) datanucleus-core-3.0.9.jar
         i) datanucleus-rdbms-3.0.8.jar
         j) commons-dbcp-1.4.jar
         k) commons-pool-1.5.4.jar
         l) antlr-runtime-3.4.jar
         m) datanucleus-api-jdo-3.0.7.jar

    HCatalog Job:
         a) above Hive required jar files
         b) hcatalog-core-0.11.0.jar

Upvotes: 1

Akash Yadav
Akash Yadav

Reputation: 2421

The problem is that org/apache/hadoop/hive/metastore/api/MetaException class is not available to java runtime. can you ensure that the Jar is under valid runtime classpath.

few try outs can be try passing the classpath explicitly to jvm via -cp , alternatively you can try putting the jar in launch directory to ensure its present.

Upvotes: 1

Related Questions