CODEWITHSUNDEEP
apache-kafka-connect
surya rahul
surya rahul

Reputation: 883

How to change the "kafka connect" component port?

On port 8083 I am running Influxdb for which I am even getting the GUI on http://localhost:8083

Now come to kafka, Here I am referring the setup as per https://kafka.apache.org/quickstart

starting the zookeeeper which is in folder /opt/zookeeper-3.4.10 by the command: bin/zkServer.sh start

So zookeeper is started now starting kafka under /opt/kafka_2.11-1.1.0 folder as :
bin/kafka-server-start.sh config/server.properties

create a topic named "test" with a single partition and only one replica:
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

Topic is created and can be checked in with command :
bin/kafka-topics.sh --list --zookeeper localhost:2181

Uptill here everything is fine and tuned.

Now I need to use "Kafka connect" component to import/export data. So I am creating a seed data as: echo -e "foo\nbar" > test.txt

Now using connector configuration for "kafka connect" to work : bin/connect-standalone.sh config/connect-standalone.properties config/connect-file-source.properties config/connect-file-sink.properties

after running above command I am getting : Address already in use

Kafka connect has stopped

enter image description here

enter image description here

I even changed the rest.port=8084 in the /opt/kafka_2.11-1.1.0/config/connect-distributed.properties so as it don't get conflict with influxdb which already on 8083. Still I am getting the same Address already in use, Kafka connect has stopped as shown in screenshots.

Upvotes: 5

Views: 21735

Answers (5)

Keet Sugathadasa
Keet Sugathadasa

Reputation: 13502

Kafka Connect Standalone mode, uses Port 8084 as the Rest API post, by default. Due to this reason, if someone else is using that port already, the process with throw a BindException.

To change the port used above, navigate to the config/connect-standalone.properties file in the Kafka Root directory.

Add the following key value property to change the Port being used for Rest API opening. (Kafka should have included this in the properties file by default, else many developers go nuts trying to find the port mapping used in the standalone mode). Put a different port as you wish.

rest.port=11133

Upvotes: 0

Nanomachines Son
Nanomachines Son

Reputation: 131

UPDATE

If you still have no luck after putting 'listensers' in connect-standalone.properties, this is what worked for me. Instead of mentioning both protocols(http & https) in the listeners value, try putting only one protocol like 'http' there and instead of writing 'localhost' as hostname, write '127.0.0.1'. So the value of listeners would be:-

listeners=http://127.0.0.1:8084

Upvotes: 1

sramu
sramu

Reputation: 1091

Kafka 3.0.0

Since Kafka Connect is intended to be run as a service, it also provides a REST API for managing connectors. The REST API server can be configured using the listeners configuration option. This field should contain a list of listeners in the following format: protocol://host:port,protocol2://host2:port2. Currently supported protocols are http and https. For example: listeners= http://localhost:8080,https://localhost:8443 By default, if no listeners are specified, the REST server runs on port 8083 using the HTTP protocol.

More details: https://kafka.apache.org/documentation/#connect_rest

Upvotes: 1

Robin Moffatt
Robin Moffatt

Reputation: 32050

Since you're using Kafka Connect in Standalone mode, you need to change the REST port in config/connect-standalone.properties: 

rest.port=18083

To understand more about Standalone vs Distributed you can read the doc here.

Upvotes: 7

jspcal
jspcal

Reputation: 51894

Change the port definition in config/server.properties:

# The port the socket server listens on
port=9092

Upvotes: -3

Related Questions