CODEWITHSUNDEEP
apache-kafkaapache-kafka-connectdebezium
lestersconyers
lestersconyers

Reputation: 382

Run Kafka Worker Process as a service

I have Kafka running with a Sql connector. Currently this is running on a Linux server and I am using a Putty connection to configure and run everything. I can start the worker process in standalone mode fine and everything works as expected. However, it feels like I should be able to leave my terminal session and keep the worker running like a service. Currently I end my terminal session and just reconnect with putty but again, this doesn't feel like the right approach. Does anyone know how to get the worker to run like a service?

./connect-standalone.sh '../config/worker.properties' '../config/connector.properties'

Upvotes: 1

Views: 1065

Answers (1)

Achyut Vyas
Achyut Vyas

Reputation: 501

First I would like to tell you that, running Kafka-connect in a standalone mode is not a good choice, and you can also run Kafka-connect in distributed mode on a single machine.

If you don't want to create a service then you can use the screen utility.

Ex.

  1. Create a Screen

    screen -S kafka-connect

  2. Run kafka-connect command

    ./connect-standalone.sh '../config/worker.properties' '../config/connector.properties'

  3. Detech the screen using ctrl+A+D

  4. List and Resume screen

    screen -ls

    screen -r kafka-connect

  5. Type exit inside the screen to the terminal attached screen.

Service

Create a new file kafka-connect.service inside /etc/systemd/system directory.

kafka-connect.service

[Unit]
Description=Kakfka-connect
After=network.target

[Service]
User=ubuntu
Group=ubuntu
Environmet="KAFKA_HEAP_OPTS=-Xmx1G -Xms1G"
Environment="KAFKA_OPTS=-javaagent:/home/ubuntu/prometheus/jmx_prometheus_javaagent-0.15.0.jar=8080:/home/ubuntu/prometheus/kafka-connect.yml"
ExecStart=/home/ubuntu/kafka_2.13-2.7.0/bin/connect-distributed.sh /home/ubuntu/kafka_2.13-2.7.0/config/connect-distributed.properties

[Install]
WantedBy=multi-user.target

In ExecStart that is the command to start Kafka-connect service as distributed mode, you can change that also If you haven't install jmx_exporter then you can remove Environment="KAFKA_OPTS=-javaagent:/home/ubuntu/prometheus/jmx_prometheus_javaagent-0.15.0.jar=8080:/home/ubuntu/prometheus/kafka-connect.yml" this line from service.

Upvotes: 2

Related Questions