Kostas Demiris
Kostas Demiris

Reputation: 3611

Hot pod reconfiguration on Kubernetes

I have deployed Prometheus on Kubernetes and I provide the prometheus.yml configuration file as a config map resource. The file is mounted in the Prometheus pod as volume.

After the config map is changed in the cluster, I hit the Prometheus server endpoint with an empty POST request, in order to reload it (as described in the documentation)

When I make changes to the config map however and redeploy it, I experience a 'lag' of around 30 secs until the prometheus.yml file is updated inside the pod.

I read here that this is expected.

However, there are projects that try to remedy this, eg

These, as I understand, kill the pods and replace them in order to update the configuration.

My question is, is there a way to make a 'hot' reconfiguration?

Essentially speed up on-demand the volume update inside the pod, without the need of killing any pod.

Upvotes: 1

Views: 478

Answers (1)

wolmi
wolmi

Reputation: 1709

I'm using right now reloader and the pods are killed always but they are recreated immediately after you change a configmap associated.

But you cannot avoid the fact that you need to kill the pod to be able to re-mount the volume with the new config and also it depends on your workload if is able to load a new configuration without stopping the main process inside the container.

Upvotes: 1

Related Questions