CODEWITHSUNDEEP
elasticsearch
Yassir S
Yassir S

Reputation: 1042

ElasticSearch: optimise the storage in indexes and the time response for requests

In a Kafka server I have N types of messages, one for each IOT application. I want to store these messages in Elastisearch in different indexes. Do you know which is the most optimizing method for that use case in order to have the lower time response for request regarding every message type ?

Furthermore, it is adivised to create an index per day like this: "messageType-%{+YYYY.MM.dd}"; Is this a way for my use case?

Finally, concerning the previous way, if I have a request with a time range for instance from 2016.06.01 to 2016.07.04, does elasticsearch search directly in the indexes "messageType-%{+2016.06.01}", "messageType-%{+2016.06.02}", ..., "messageType-%{+2016.07.04}" ?

Thanks in advance,

J

Upvotes: 0

Views: 42

Answers (1)

codegrep_admin
codegrep_admin

Reputation: 529

If you plan to purge docs after a certain time, creating indexes based on time is a good idea because you can drop indexes after certain time.

You can search against all indexes or more preferably you should specify the indexes you want to search against.

For example, you could do a search against /index1,index2/_search where you determine index1, index2 from the query or you can just hit /_search which will search all indexes (slower)

Upvotes: 1

Related Questions