CODEWITHSUNDEEP
apache-kafkaapache-kafka-connect
Artanis
Artanis

Reputation: 631

Kafka and Kafka Connect deployment environment

if I already have Kafka running on premises, is Kafka Connect just a configuration on top of my existing Kafka, or does Kafka Connect require it's own Server/Environment separate from that of my existing Kafka?

Upvotes: 1

Views: 743

Answers (2)

Robin Moffatt
Robin Moffatt

Reputation: 32090

Kafka Connect is part of Apache Kafka, but it runs as a separate process, called a Kafka Connect Worker. Except in a sandbox environment, you would usually deploy it on a separate machine/node from your Kafka brokers.

This diagram shows conceptually how it runs, separate from your brokers:

Diagram of Kafka Connect

You can run Kafka Connect on a single node, or as part of a cluster (for throughput and redundancy).

You can read more here about installation and configuration and architecture of Kafka Connect.

Upvotes: 3

pacman
pacman

Reputation: 1061

Kafka Connect is its own configuration on top of your bootstrap-server's configuration.

For Kafka Connect you can choose between a standalone server or distributed connect servers and you'll have to update the corresponding properties file to point to your currently running Kafka server(s).

Look under {kafka-root}/config and you'll see enter image description here

You'll basically update connect-standalone or connect-distributed properties based on your need.

Upvotes: 0

Related Questions