CODEWITHSUNDEEP
databasemongodbelasticsearchweb-scrapingfull-text-search
SanthoshSolomon
SanthoshSolomon

Reputation: 1402

Elastic Search as Persistent database

I am trying to understand the feasibility of utilising Elastic Search as persistent storage. My purpose would be, high number of write actions - from scraping multiple scraping instances.

Currently I am having Mongo DB as storage and now I have to index the data in Mongo DB for making a text search application. To avoid using multiple storage instances .i.e. Mongo DB and ES, I am trying to get a clear picture about this point.

On searching through internet, most of the articles are way old (dated 2011-2015) and suggesting not to use ES as persistent storage.

Thanks in advance.

Upvotes: 2

Views: 2415

Answers (1)

Jaycreation
Jaycreation

Reputation: 2089

Usually, Elasticsearch, as a non relational DB is not enough to respond to all use cases of a application. That's why Elastic is generally used over another SGBD (postgres, Oracle, etc...) For High volume of write, clients generally use Cassandra has data lake and elastic search to visualization on part of data. Frequent updates on part of data can be very complicated to manage, and designing effective indices in that goal can be challenging.

Now, if you ensure to have a cluster with at least 1 replica on all your indices, and create regular snapshots, your data are as secured in Elasticsearch as in another well managed database. And you can ensure even more security with a cross cluster backup, several 'zone' for one cluster, etc...

Furthermore, an elastic cluster can scale well and manage easily indices of several hundreds of GB.

It's always complicated to choose a SGBD, and each use case is different, but if your application can do everything with Elasticsearch queries, you can use only Elasticsearch. (in full search cases I just don't need any DB call after ingesting my docs)

Upvotes: 4

Related Questions