CODEWITHSUNDEEP
apache-kafkaapache-zookeeperstrimzi
Sunny Kanda
Sunny Kanda

Reputation: 1

Clone a Kafka / ZooKeeper cluster for testing

I have started at a new company, and they only have two environments, one is a test environment and the other is a Prod environment. In the test environment, lots of development teams are using this single instance of Kafka.

I want to upgrade Strimzi, which manages Kafka, and eventually I want to upgrade Kafka too. However, I want to test my script first, and I also want a way to restore the current state incase I break anything.

I have taken a copy of all the yaml in the namespace, but not sure if just re-applying this would restore anything I break. I was hoping there was a way to just clone the entire setup?

Also, I want to create a new test environment, for unit testing, a bit of a route to live, but not sure how to do this.

I have tried to copy and save all the YAML and deploy to a local Docker container, but this doesnt work. I havent tried anything in regards to the upgrade failing. Upgrading on the local Docker container is working fine

Upvotes: 0

Views: 57

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191671

clone the entire setup?

Short answer - Install and use tools like Velero. Don't manually copy YAML files. Ideally, you should use Gitops already, and not apply your resource files without version control.

You'll obviously need to backup any PersistentVolume resources for all brokers, Zookeeper nodes, etc. as a well. When restoring, the node.id's must be the same...

Otherwise, you could also deploy MirrorMaker2 in your "new environment" then have it actively consume any data that you wish to copy (assuming you don't really need all of it)

create a new test environment, for unit testing

Sounds like you are asking for integration testing. We don't need a broker for unit testing; use mocks.

Upvotes: 0

Related Questions