CODEWITHSUNDEEP
apache-kafkaconfluent-schema-registry
Zach Larson
Zach Larson

Reputation: 47

Confluent.Kafka Consumer Schema Registry artifact lookup error

I am getting an error No artifact with ID '33' was found.; error code: 404 when i call Consume() on my consumer. However, if I make a direct call to schemaRegistry.GetLatestSchemaAsync($"{TOPIC}-value") I get the correct schema with ID: 39. Why would the consumers use of my schema registry end up with a different artifact ID? Not sure if it's a caching issue or an artifact version issue.

given this code (lots omitted to keep it short):

const string TOPIC = "topic-01";

using (var schemaRegistry = new CachedSchemaRegistryClient(schemaRegistryConfig))
using (var consumer = new ConsumerBuilder<string, MyCustomClass>(config)
         .SetValueDeserializer(new AvroDeserializer<MyCustomClass>(schemaRegistry).AsSyncOverAsync())
{
  //this returns me the details of this subject with an ID: 39.  This is the ID I would expect
  var latest = await schemaRegistry.GetLatestSchemaAsync($"{TOPIC}-value");

  try
  {
    var consumeResult = consumer.Consume(cts.Token);
  }
  catch (Exception e)
  {
     //throws: No artifact with ID '33' was found.; error code: 404
  }
}```

Upvotes: 0

Views: 554

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 192023

A subject in the registry can have multiple versions, each version having a different ID.

There's no way to bypass the ID lookup procedure in the consumer deserializer such that you only get a hard-coded schema or ID; it'll always attempt to use the ID within the message

If you manually look at each /subjects/:name/versions endpoint, you'll probably find that the ID in the error indeed doesn't exist, which could mean somebody deleted it and you'll need to skip past all the offsets that could've used that schema version

Upvotes: 2

Related Questions