CODEWITHSUNDEEP
apache-kafkaconfluent-platformconfluent-schema-registryksqldb
Oles Rid
Oles Rid

Reputation: 135

Troubles with ksql running in docker

I have confluent kafka, zookeeper, schema-registry and ksql running in containers on Kubernetes cluster. Kafka, zookeeper and schema registry works fine, a can create topic and write data in Avro format, but when I'm trying to check ksql and create some streaming with curl like:

curl -XPOST http://ksql-svc.someapp:8080/ksql -H "Content-Type: application/json" -d $' 
{"ksql": "CREATE STREAM kawabanga_stream (log_id varchar, created_date varchar) WITH (kafka_topic = '\'kawabanga\'', value_format = '\'avro\'');","streamsProperties":{}}'

I get error:

[{"error":{"statementText":"CREATE STREAM kawabanga_stream (log_id varchar, created_date varchar) WITH (kafka_topic = 'kawabanga', value_format = 'avro');","errorMessage":{"message":"Avro schema file path should be set for avro topics.","stackTrace":["io.confluent.ksql.ddl.commands.RegisterTopicCommand.extractTopicSerDe(RegisterTopicCommand.java:75)","io.confluent.ksql.ddl.commands.RegisterTopicCommand.<init>

Please find belowe my ksql server config:

# cat /etc/ksql/ksqlserver.properties
bootstrap.servers=kafka-0.kafka-hs:9093,kafka-1.kafka-hs:9093,kafka-
2.kafka-hs:9093
schema.registry.host=schema-svc
schema.registry.port=8081
ksql.command.topic.suffix=commands
listeners=http://0.0.0.0:8080

Also I tried to start server without schema.registry strings but with no luck

Upvotes: 0

Views: 587

Answers (2)

miguno
miguno

Reputation: 15057

You must set the configuration ksql.schema.registry.url (see KSQL v0.5 documentation).

FYI: We will have better documentation for Avro usage in KSQL and for Confluent Schema registry integration with the upcoming GA release of KSQL in early April (as part of Confluent Platform 4.1).

I don't know how to check version, but I'm using docker image confluentinc/ksql-cli (I think it's 0.5). For Kafka I used Confluent Kafka Docker image 4.0.0

  • The KSQL server shows the version on startup.
  • If you can't see the KSQL server startup message, you can also check the KSQL version by entering "VERSION" in the KSQL CLI prompt (ksql> VERSION).

Upvotes: 2

Andrew Coates
Andrew Coates

Reputation: 1893

According to the KSQLConfig class, you should use the ksql.schema.registry.url property to specify the location of the Schema Registry.

This looks to of been the case since at least v0.5 onwards.

It's also worth noting that using the RESTful API directly isn't currently supported. So you may find the API changes between releases.

Upvotes: 1

Related Questions