CODEWITHSUNDEEP
high-availabilityprometheus
Nipun Talukdar
Nipun Talukdar

Reputation: 5387

How can we get high availability in prometheus data store?

I am new to prometheus, and so I am not sure if high availability is part of Prometheus data store tsdb. I am not looking into something like having two prometheus server instances scraping data from the same exporter as that has high chance of having two tsdb data store which are out of sync.

Upvotes: 4

Views: 4149

Answers (3)

Niksa Jakovljevic
Niksa Jakovljevic

Reputation: 99

There is prometheus-postgresql-adapter that allows you to use PostgreSQL / TimescaleDB as a remote storage. The adapter enables multiple Prometheus instances (HA setup) to write to a single remote storage, so you have one source of truth. Recently, I've published a blog post about it [How to manage Prometheus high-availability with PostgreSQL + TimescaleDB] (https://blog.timescale.com/blog/prometheus-ha-postgresql-8de68d19b6f5/).

Disclaimer: I am one of the engineers behind the adapter

Upvotes: -1

Mohamed chiheb Ben jemaa
Mohamed chiheb Ben jemaa

Reputation: 546

For Kubernetes setup Using kube-prometheus (prometheus-operator), you can configure it using values.

and including thanos Would help in this situation

Upvotes: 0

remeika
remeika

Reputation: 1155

It really depends on your requirements.

Do you need highly available alerting on your metrics? Prometheus can do that.

Do you need a highly available monitoring system that contains the last few hours of data for operational triage? Two prometheus instances are pretty good for that too.

Do you need long-term storage of timeseries data? Prometheus is not designed to accomplish this on its own. Either use the remote write functionality of prometheus to ship data to another TSDB that supports redundant storage (InfluxDB and Clickhouse are pretty promising here) but you are on the hook for de-duping data. Alternatively, consider Cortex.

Upvotes: 10

Related Questions