Not able to start Kafka-Connect as a service on CentOS 7

Question

I have a Kafka environment (Zookeeper + Kafka Server + Kafka-Connect) which runs perfectly when I use command line to start each individual components on CentOS 7.

Now I am setting up these Kafka components to run as a service. For this I have created .service files and placed it in /etc/systemd/system folder. Following are the files

zookeeper.service

#!/bin/bash
# vi /etc/systemd/system/zookeeper.service
[Unit]
Description=This service will start Zookeeper server which will be used by Kafka Server. 
Requires=network.target remote-fs.target
After=network.target remote-fs.target

[Service]
Type=simple
ExecStart=/opt/interactcrm/kafka_2.11-1.0.1/bin/zookeeper-server-start.sh /opt/interactcrm/kafka_2.11-1.0.1/config/zookeeper.properties
ExecStop=/opt/interactcrm/kafka_2.11-1.0.1/bin/zookeeper-server-stop.sh
TimeoutStartSec=0
Restart=on-abnormal

[Install]
WantedBy=multi-user.target

kafka.service

#!/bin/bash
# vi /etc/systemd/system/kafka.service
[Unit]
Description=This service will start Kafka server. 
Requires=zookeeper.service
After=zookeeper.service

[Service]
Type=simple
ExecStart=/opt/interactcrm/kafka_2.11-1.0.1/bin/kafka-server-start.sh /opt/interactcrm/kafka_2.11-1.0.1/config/server.properties
ExecStop=/opt/interactcrm/kafka_2.11-1.0.1/bin/kafka-server-stop.sh
TimeoutStartSec=0
Restart=on-abnormal

[Install]
WantedBy=multi-user.target

Kafka-connect.service

#!/bin/bash
# vi /etc/systemd/system/kafkaconnect.service
[Unit]
Description=This service will start Kafka Connect Service. 
Requires=network.target remote-fs.target nss-lookup.target kafka.service kafka.service
After=network.target remote-fs.target nss-lookup.target kafka.service 


[Service]
Type=forking
Environment="KAFKA_JMX_OPTS=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=10040 -Dcom.sun.management.jmxremote.local.only=true -Dcom.sun.management.jmxremote.authenticate=false"
Environment="LOG_DIR=/var/log/kafka-logs"
ExecStart=/opt/interactcrm/kafka_2.11-1.0.1/bin/connect-distributed.sh /opt/interactcrm/kafka_2.11-1.0.1/config/connect-distributed.properties
TimeoutStartSec=1000
#Restart=on-abnormal
#SuccessExitStatus=143

[Install]
WantedBy=multi-user.target

Zookeeper and Kafka services starts without any issue. I can create topics and then do operations on the topic. The issue is with Kafka connect service.

When I try to start the service using systemctl command, the service does not start. It gets stuck no following log ::

Oct 19 18:29:20 localhost.localdomain connect-distributed.sh[1071]: [2018-10-19 18:29:20,713] INFO Added plugin 'io.debezium.connector.mysql.MySqlConnector...er:136)
Oct 19 18:29:20 localhost.localdomain connect-distributed.sh[1071]: [2018-10-19 18:29:20,713] INFO Added plugin 'io.debezium.transforms.ByLogicalTableRoute...er:136)
Oct 19 18:29:20 localhost.localdomain connect-distributed.sh[1071]: [2018-10-19 18:29:20,713] INFO Added plugin 'io.debezium.transforms.UnwrapFromEnvelope'...er:136)
Oct 19 18:29:20 localhost.localdomain connect-distributed.sh[1071]: [2018-10-19 18:29:20,761] INFO Loading plugin from: /opt/interactcrm/debezium/debezium ...er:184)
Oct 19 18:29:28 localhost.localdomain connect-distributed.sh[1071]: [2018-10-19 18:29:28,725] INFO Registered loader: PluginClassLoader{pluginLocation=file...er:207)

I cannot find any log for this process in message logs after this line and there is no error in any other logs. The process gets stuck on this line ::

INFO Registered loader: PluginClassLoader{pluginLocation=file...er:207)

No matter how much I increase the timeout this process never starts. But when I run the same command from command line, the service starts properly.

I have tried to remove all connectors from Plugin path to see if the service start but it gets stuck on the same line.

Following is my reference point ::

Kafka-Connect Service

Answer 1

I faced the same problem on Debain 9. Figure it out it was because the service need a WorkingDirectory otherwise kafka-connect never fully charges.

So your service should look like this:

#!/bin/bash
# vi /etc/systemd/system/kafkaconnect.service
[Unit]
Description=This service will start Kafka Connect Service. 
Requires=network.target remote-fs.target nss-lookup.target kafka.service kafka.service
After=network.target remote-fs.target nss-lookup.target kafka.service 


[Service]
Type=forking
Environment="KAFKA_JMX_OPTS=-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=10040 -Dcom.sun.management.jmxremote.local.only=true -Dcom.sun.management.jmxremote.authenticate=false"
Environment="LOG_DIR=/var/log/kafka-logs"
WorkingDirectory="/opt/interactcrm/kafka_2.11-1.0.1" <--- or whatever directory you to use
ExecStart=/opt/interactcrm/kafka_2.11-1.0.1/bin/connect-distributed.sh /opt/interactcrm/kafka_2.11-1.0.1/config/connect-distributed.properties
TimeoutStartSec=1000
#Restart=on-abnormal
#SuccessExitStatus=143

[Install]
WantedBy=multi-user.target

Answer 2

** Below configuration worked for me in Ubuntu **

[Unit]
Requires=kafka.service
After=kafka.service

[Service]
Type=simple
User=kafka
ExecStart=/bin/sh -c '/home/kafka/kafka/bin/connect-distributed.sh /home/kafka/kafka/config/connect-distributed.properties > /home/kafka/kafka/kafka_connect.log 2>&1'
Restart=on-abnormal

[Install]
WantedBy=multi-user.target