CODEWITHSUNDEEP
apache-kafkastreamingconfluent-platformconfluent-schema-registryksqldb
Ivan Rubanau
Ivan Rubanau

Reputation: 384

ksqlDB can't get data from Schema Registry

Case: I have topic in Kafka with name some_table_prefix.table_name. Data is serialized with AVRO, but for historical reasons I have record in Schema Registry named table_name-value.

When I'm trying to setup ksqlDB stream

CREATE STREAM some_stream_name
WITH (KAFKA_TOPIC='some_table_prefix.table_name', VALUE_FORMAT='AVRO');

I'm getting error Schema for message values on topic some_table_prefix.table_name does not exist in the Schema Registry.Subject: some_table_prefix.table_name-value.

I have Schema registry integrated correctly, for others topics everything works ok.

So, is it possible to specify Schema Registry record name in ksqlDB stream creation or resolve this issue some other way?

Upvotes: 1

Views: 1012

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191671

If you have a topic named table_name, that has Avro being produced to it (which would automatically create table_name-value in the Registry), then that's what ksqlDB should consume from. If you'd manually created that subject by posting the schema on your own, without matching the topic name, then that's part of the problem.

As the error says, it's looking for a specific subject in the Registry based on the topic you've provided. To my knowledge, its not possible to use another subject name, so the workaround is to POST the old subject's schemas into the new one

Upvotes: 1

Related Questions