CODEWITHSUNDEEP
javahive
Alex Woolford
Alex Woolford

Reputation: 4563

Hive UDF class not found

I need an auto-incrementing integer column in Hive and stumbled across UDFRowSequence.

I created a Maven project in IntelliJ, added the .java file, and let the IDE import the dependencies. I then ran mvn package, copied the resulting .jar to a cluster node, and added the .jar resource in Hive:

hive> add file udf-row-sequence-1.0-SNAPSHOT.jar;
Added resources: [udf-row-sequence-1.0-SNAPSHOT.jar]

Unfortunately, I'm unable to create the temporary function:

hive> create temporary function row_sequence as 'com.alexwoolford.hive.udf.UDFRowSequence';
FAILED: Class com.alexwoolford.hive.udf.UDFRowSequence not found
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.FunctionTask

And yet, if I look at the contents of the .jar file I can see that the class exists:

jar tf udf-row-sequence-1.0-SNAPSHOT.jar
[...]
com/alexwoolford/hive/udf/UDFRowSequence.class
[...]

Can you see what I'm doing wrong?

Upvotes: 1

Views: 2807

Answers (2)

Gaurav Khare
Gaurav Khare

Reputation: 2327

Even though you do "ADD JAR", sometimes even order of adding the JARS do matter, I faced this issue while adding ESRI Hive UDF jars.

add jar esri-geometry-api-1.2.jar;
add jar spatial-sdk-hive-1.0.3-SNAPSHOT.jar;

works

add jar spatial-sdk-hive-1.0.3-SNAPSHOT.jar;
add jar esri-geometry-api-1.2.jar;

Doesn't work

Upvotes: 0

Alex Woolford
Alex Woolford

Reputation: 4563

The issue was caused by using add file instead of add jar, i.e.

add jar udf-row-sequence-1.0-SNAPSHOT.jar;

Upvotes: 1

Related Questions