Both ElasticSearch and Redis, overkill usecase?

Question

I'm currently designing the architecture of my project and trying to decide which data store to use.

Simple use case: I have several thousand profiles in a backend and I need to implement a fast search engine. Elasticsearch looked perfect for that case: every time a profile is updated, the index will be updated by an asynchronous task.

Additional information:

• profiles updated at least once a week
• 500k - 1 million users

My question: If I want to implement a cache system for profile details, should I stick with Elasticsearch and put the data in my index? Or should I use Redis and do something like profil_id => data ?

I think both sound good; the problem is whenever a profile is updated, I will have to flush it after the reindexing in Elasticsearch if I want to see the change in my backend.

So what can I do?

Answer 1

You should consider using RediSearch. Using RediSearch can provide you a solution for your needs, getting both Redis performance and a full-text support.

Answer 2

Elasticsearch and redis are basically meant to solve two different problems, As one does indexing while other does caching. Redis is meant to return already requested data as fast as possible whereas as Elasticsearch is a search and analytics engine, it would perfectly fit a use-case where you have to implement a fast search engine and it will be more performant than any in-memory data structure store or cache such as redis(Assuming your searches will be complex, will involve some aggregation/filters).

The problem comes profile updates Since your profile updates are not that frequent you could actually do partial updates to the ES index rather doing reindex.So whenever a person updates its profile get the changeling set(changed data) and do a partial update to the particular document in ES Index. You can see how its done here partial update.

This one particular stackoverflow answer will help you cache vs indexing