CODEWITHSUNDEEP
elasticsearchdata-warehouseamazon-redshift
Yannis
Yannis

Reputation: 6157

Why people often use a database like Redshift together with an ElasticSearch purely for analytics / reporting queries?

Per the title - I have seen that many companies - especially in ad tech - use a data warehouse solution like Redshift, where they store all the transactional data to do aggregations and analytics, and also pump their data in elastic search for possibly the same reason (not for search anyways).

Apologies if this questions looks daft but wanted to understand the reasons behind this.

Is it to get real-time queries out of one and do historical data analysis on the other?

Thanks

Upvotes: 0

Views: 1450

Answers (1)

denismo
denismo

Reputation: 800

Indeed, I've worked with a few companies (as a consultant) who were considering a combination of these 2 exactly for the similar reasons to what you described:

  • Redshift: for historical analysis, large complex queries, joins, trends, pre-aggregations

  • ElasticSearch (usually with Kibana): for near real-time operational monitoring and analytics, leveraging its indexing capabilities and free-form searches, dashboards, JSON indexing, real-time metric queries

Redshift is great for handling massive amounts of time-series data (billions of rows in seconds). But it's not ideal for frequent queries on real-time streamed data, and that's where ElasticSearch comes in.

Upvotes: 1

Related Questions