npt
npt

Reputation: 11

Debezium with custom Driver for mysql 5.1

I'm new to debezium CDC and I tested using debezium-server to stream data from mysql 5.5 to pubsub sink. I know the version of mysql is very old and not officially supported based on debezium documentation but I need to use this version.

I deployed debezium-server on local machine with Docker Compose as below. This works perfectly find. Debezium-server version is 2.6.0.Final, I can see this driver file in /lib/mysql-connector-j-8.2.0.jar.

# docker-compose.yaml
services:
  debezium-server:
    image: quay.io/debezium/server:2.6.0.Final
    container_name: debezium-server
    ports:
      - 8080:8080
    environment:
      - PUBSUB_EMULATOR_HOST=pubsub:8085
      - PUBSUB_PROJECT_ID=abc
    volumes:
      - $PWD/debezium-server/conf:/debezium/conf
      - $PWD/debezium-server/data:/debezium/data
      - $PWD/debezium-server/lib/mysql-connector-java-5.1.40-bin.jar:/debezium/lib/mysql-connector-java-5.1.40-bin.jar # try this which gives "class not found" error
    depends_on:
      - postgres
      - pubsub

  pubsub:
    container_name: pubsub
    image: gcr.io/google.com/cloudsdktool/google-cloud-cli:414.0.0-emulators
    ports:
      - 8085:8085
    command: 
    - gcloud 
    - beta 
    - emulators 
    - pubsub 
    - start
    - --project=abc
    - --host-port=0.0.0.0:8085
# application.properties

# Source Configuration --
debezium.source.connector.class=io.debezium.connector.mysql.MySqlConnector
debezium.source.offset.storage=org.apache.kafka.connect.storage.FileOffsetBackingStore
debezium.source.offset.storage.file.filename=data/offsets.dat
debezium.source.schema.history.internal=io.debezium.storage.file.history.FileSchemaHistory
debezium.source.schema.history.internal.file.filename=data/schemahistory.dat

debezium.source.database.hostname=${HOST}
debezium.source.database.port=3306
debezium.source.database.user=${USER}
debezium.source.database.password=${PASSWORD}
debezium.source.database.dbname=${DB}
debezium.source.database.server.id=1
debezium.source.tombstones.on.delete=false

# Sink Configuration --
debezium.sink.type=pubsub
debezium.sink.pubsub.project.id=abc
debezium.sink.pubsub.address=pubsub:8085


#Quarkus Configuration --
quarkus.log.console.json=false

Now when I tried streaming with mysql 5.1, debezium breaks with CLIENT_PLUGIN_AUTH error. I assumed this is because the driver is not compatible with mysql 5.1. When I tried mounting mysql-connector-driver-5.1.40-bin.jar to /lib/, debezium breaks with "java class not found" error. Is there a way to use older version of mysql driver to make debezium-server compatible with mysql 5.1? Thank you.

Upvotes: 0

Views: 39

Answers (0)

Related Questions