Reputation: 3611
I have deployed Prometheus on Kubernetes and I provide the prometheus.yml
configuration file as a config map resource.
The file is mounted in the Prometheus pod as volume.
After the config map is changed in the cluster, I hit the Prometheus server endpoint with an empty POST request, in order to reload it (as described in the documentation)
When I make changes to the config map however and redeploy it, I experience a 'lag' of around 30 secs until the prometheus.yml
file is updated inside the pod.
I read here that this is expected.
However, there are projects that try to remedy this, eg
These, as I understand, kill the pods and replace them in order to update the configuration.
My question is, is there a way to make a 'hot' reconfiguration?
Essentially speed up on-demand the volume update inside the pod, without the need of killing any pod.
Upvotes: 1
Views: 478
Reputation: 1709
I'm using right now reloader and the pods are killed always but they are recreated immediately after you change a configmap associated.
But you cannot avoid the fact that you need to kill the pod to be able to re-mount the volume with the new config and also it depends on your workload if is able to load a new configuration without stopping the main process inside the container.
Upvotes: 1