CODEWITHSUNDEEP
hadoopclouderaavro
Emre Sevinç
Emre Sevinç

Reputation: 8511

What happened to the AvroRecord class in CDH 5?

I'm trying to use CDH 5 (Cloudera Hadoop Distribution) repositories in my pom.xml as described in Cloudera documentation. But Maven complains about AvroRecord not found.

Here's the repository setting from my pom.xml:

<repositories>
    <repository>
      <id>cloudera</id>
      <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
    </repository>
</repositories>

And here's how I declare the dependency to Hadoop in my pom.xml (the exclusion part is about related to getting rid of the eror "javax.servlet.FilterRegistration's signer information does not match signer information of other classes in the same package"):

   <dependency>
      <groupId>org.apache.hadoop</groupId>
      <artifactId>hadoop-common</artifactId>
      <version>2.5.0-cdh5.2.1</version>
      <scope>provided</scope>
      <exclusions>
        <exclusion>
          <groupId>javax.servlet</groupId>
          <artifactId>*</artifactId>
        </exclusion>
      </exclusions>
    </dependency>

When I check the documentation for Hadoop, AvroRecord seems to exist:

http://hadoop.apache.org/docs/r2.5.0/hadoop-project-dist/hadoop-common/api/org/apache/hadoop/io/serializer/avro/AvroRecord.html

But when I check the source code of Cloudera for branch cdh5-2.5.0_5.2.1, it does not seem to exist:

https://github.com/cloudera/hadoop-common/tree/cdh5-2.5.0_5.2.1/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/serializer/avro

In other words, the following seems problematic with Cloudera version but seemed to work when I used the Hadoop repository:

import org.apache.hadoop.io.serializer.avro.AvroRecord;

How can I get the AvroRecord class when I use Cloudera Hadoop?

Upvotes: 0

Views: 113

Answers (1)

Sean Owen
Sean Owen

Reputation: 66886

This took me a minute to figure out. It's "not there" upstream too: https://github.com/apache/hadoop/tree/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/serializer/avro

But this is because it's a generated class. The definition is here: https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/test/avro/avroRecord.avsc https://github.com/cloudera/hadoop-common/blob/cdh5-2.5.0_5.2.1/hadoop-common-project/hadoop-common/src/test/avro/avroRecord.avsc

You're not finding it presumably because it's a test class too, not included in any distribution. What I'm slightly confused about is how it got into the main project javadoc.

Upvotes: 1

Related Questions