CODEWITHSUNDEEP
docker-composemetricsgrafanaprometheus
nagrom97
nagrom97

Reputation: 514

How to monitor / discover more docker containers with prometheus

I am using prometheus, node-exporter and grafana locally with docker-compose to see how metrics can be displayed. Is it possible to add a container as a service (such as apache \ nginx ) in the docker-compose.yml and have the service discovered so that its metrics are also taken into account? The documentation is not too helpful in explaining this.

Upvotes: 0

Views: 2031

Answers (1)

FuzzyAmi
FuzzyAmi

Reputation: 8109

If everything needs to run locally (ie, both your Prometheus and the monitored entities are on the same machine), I would not bother with proper auto-discovery. Rather, I'd just have a static config file in Prometheus listing each of the services it needs to monitor.

Each of these services would be running in a docker container, each with its own metric-exporting app (like the node-exporter or Telegraf). Yes, this means you'll need to extend your nginx-docker with telegraf (or other tool) to have a metric-emitting docker container.

that way, Prometheus would be scraping n urls for n services. You could probably run all them in --net=host mode, but then each service would need to export its metrics in a different port.

Keep in mind that when you gather metrics from within a docker container, some of metrics might be wrong, since the container has a limited view of the system. So OS/HW metrics might be off. But if you're interested in custom/app-level metrics, that should be fine.

so, to sum: - each docker needs to run its own exporter (maybe exporting in a different port) - prometheus should be configured to know where to look for each of these services.

if you really want auto-discovery, look into Consul and how it integrates with Prometheus. I'm told you can actually run consul-server AND consul-clients on the same machine, and with it you can setup Prometheus to read the list of services from Consul (meaning, have auto-discovery) without having to explicitly list each of them in the config file.

Using zookeeper should also work: your containers would register with zookeeper and Prometheus would be aware of them from zk (disclaimer: I've never tried this particular configuration).

Upvotes: 1

Related Questions