Cristian Ciobotea
Cristian Ciobotea

Reputation: 457

Apache Avro Maven Plugin generate UUID field instead of String

Given the following Avro Schema:

{
    "namespace": "ro.dspr.coreentities",
    "type": "record",
    "name": "Organization",
    "fields": [
      {
        "name": "id",
        "type": "string",
        "logicalType": "uuid"
      },
      {
        "name": "name",
        "type": "string"
      },
      {
        "name": "description",
        "type": "string"
      }
    ]
}

Running avro-maven-plugin 1.9.0 goal schema, I get:

@org.apache.avro.specific.AvroGenerated
public class Organization extends org.apache.avro.specific.SpecificRecordBase implements org.apache.avro.specific.SpecificRecord {

  public static final org.apache.avro.Schema SCHEMA$ = new org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"Organization\",\"namespace\":\"ro.dspr.coreentities\",\"fields\":[{\"name\":\"id\",\"type\":{\"type\":\"string\",\"avro.java.string\":\"String\"},\"logicalType\":\"uuid\"},{\"name\":\"name\",\"type\":{\"type\":\"string\",\"avro.java.string\":\"String\"}},{\"name\":\"description\",\"type\":{\"type\":\"string\",\"avro.java.string\":\"String\"}}]}");

  // truncated 

  @Deprecated public java.lang.String id;
  @Deprecated public java.lang.String name;
  @Deprecated public java.lang.String description;

  // truncated
}

I want the generated POJO for Organization to have id UUID, not String (what I have now).

Links I looked at:

I do see the logical type def from Avro and there is the Conversion class I am actually trying to trigger, but I cannot connect the dots.

Other

Relevant Maven pom parts

<plugin>
  <groupId>org.apache.avro</groupId>
  <artifactId>avro-maven-plugin</artifactId>
  <version>${avro.version}</version>
  <configuration>
    <sourceDirectory>${avro-files-path}</sourceDirectory>
    <outputDirectory>${project.build.directory}/generated-sources</outputDirectory>
    <stringType>String</stringType>
  </configuration>
  <executions>
    <execution>
      <goals>
        <goal>schema</goal>
      </goals>
      <configuration>
        <sourceDirectory>${avro-files-path}</sourceDirectory>
        <outputDirectory>${project.build.directory}/generated-sources</outputDirectory>
      </configuration>
    </execution>
  </executions>
</plugin>

Extra info

I actually trying to use Avro to give my Kafka messages a structure. I am also using Confluent Schema Registry and Confluent Avro Kafka Serializer. Nevertheless, I thought I would only have the id as String, but if I try to send messages to Kafka as something non-UUID, it will fail later. However, I found out that there is actually no constraint, and I managed to send any String to Kafka. So, the "logicalType" in Avro is not enforced at all.

The Question

  1. How can I generate Organization.class#id as UUID?
  2. If there is no Avro-support in doing this, what would be the workaround (preferably reusing the org.apache.avro.Conversions.UUIDConversion) ?

Upvotes: 2

Views: 3255

Answers (1)

M21B8
M21B8

Reputation: 1887

Here is a functional example:

Schema:

{
  "name": "pk",
  "type": {"type": "string", "logicalType": "uuid"}
},

Maven config:

            <plugin>
                <groupId>org.apache.avro</groupId>
                <artifactId>avro-maven-plugin</artifactId>
                <version>1.9.1</version>
                <executions>
                    <execution>
                        <phase>generate-sources</phase>
                        <goals>
                            <goal>schema</goal>
                        </goals>
                        <configuration>
                            <sourceDirectory>${project.basedir}/../avro/schemas/commands</sourceDirectory>
                            <outputDirectory>${project.build.directory}/generated-sources/java</outputDirectory>
                            <!-- TODO Enable this string type once we have 1.10.0 release, or 1.9.2-->
                            <!--<stringType>String</stringType>-->
                            <fieldVisibility>PRIVATE</fieldVisibility>
                            <customConversions>org.apache.avro.Conversions$UUIDConversion</customConversions>
                            <imports>
                                <import>${project.basedir}/../avro/schemas/common</import>
                                <import>${project.basedir}/../avro/schemas/commands/account</import>
                                <import>${project.basedir}/../avro/schemas/commands/rec</import>
                            </imports>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

The important thing to note is that there is a conflict between the UUIDConversion and String in the current release of the plugin, as such you either need to choose which is more important, or use 1.10.0-SNAPSHOT which has a fix merged, but not yet released.

Upvotes: 4

Related Questions