CODEWITHSUNDEEP
kubernetesapache-kafkadebeziumstrimzi
sammy
sammy

Reputation: 113

Debezium KafkaConnect, kafka_connect_run.sh No such file or directory

I am getting the error when attempting to setup Debezium in kubernetes (minikube for now) using Strimzi. What is wrong with my manifest? I may be confusing how to use the debezium connect image. I am new to debezium and trying to setup a local test in minikube. Thanks.

Kafka setup via Strimzi:

kubectl create ns kafka
kubectl create -f 'https://strimzi.io/install/latest?namespace=kafka' -n kafka
kubectl apply -f https://strimzi.io/examples/latest/kafka/kraft/kafka-single-node.yaml -n kafka

Error:

kafka -n kafka logs my-connect-cluster-connect-0                       
Using BOOTSTRAP_SERVERS=10.99.215.168:9092
Plugins are loaded from /kafka/connect
/docker-entrypoint.sh: line 330: /opt/kafka/kafka_connect_run.sh: No such file or directory

Kubernetes manifest:

apiVersion: v1
kind: Service
metadata:
  name: postgres
  namespace: kafka
spec:
  ports:
    - port: 5432
      targetPort: 5432
  selector:
    app: postgres
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
  namespace: kafka
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
        - name: postgres
          image: quay.io/debezium/example-postgres:2.7
          ports:
            - containerPort: 5432
          env:
            - name: POSTGRES_USER
              value: postgres
            - name: POSTGRES_PASSWORD
              value: postgres
---
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
  name: my-connect-cluster
  namespace: kafka
spec:
  version: 3.0.0
  replicas: 1
  bootstrapServers: my-cluster-kafka-bootstrap:9092
  image: quay.io/debezium/connect:latest
  resources:
    requests:
      memory: 2Gi
      cpu: 1
    limits:
      memory: 4Gi
      cpu: 2
  config:
    group.id: connect-cluster
    offset.storage.topic: connect-offsets
    config.storage.topic: connect-configs
    status.storage.topic: connect-status
    key.converter: org.apache.kafka.connect.json.JsonConverter
    value.converter: org.apache.kafka.connect.json.JsonConverter
    key.converter.schemas.enable: false
    value.converter.schemas.enable: false
---
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
  name: inventory-connector
  namespace: kafka
  labels:
    strimzi.io/cluster: my-connect-cluster
spec:
  class: io.debezium.connector.postgresql.PostgresConnector
  tasksMax: 1
  config:
    bootstrap.servers: "my-cluster-kafka-bootstrap:9092"
    database.hostname: postgres
    database.port: "5432"
    database.user: postgres
    database.password: postgres
    database.dbname: postgres
    database.server.name: postgres
    slot.name: debezium_slot
    publication.name: debezium_pub
    plugin.name: pgoutput
    topic.prefix: postgres
    snapshot.mode: initial

Upvotes: 0

Views: 149

Answers (1)

Jakub
Jakub

Reputation: 3981

The container image you use has to be based on the Strimzi Kafka image for the corresponding Strimzi (and Kafka) version. I'm not sure that is the case with quay.io/debezium/connect:latest as the error suggests.

You can check the Strimzi Docs that contain a guides for how to add connector plugins to the Strimzi image and you have to follow that.

Upvotes: 2

Related Questions