CODEWITHSUNDEEP
elasticsearchcustom-plugins
TRP
TRP

Reputation: 53

Custom decoding in Elasticsearch

I want to encode my data before putting it into Elasticsearch using dictionary encoding. The data is quite tabular, singular values, a lot of duplicates in columns, so dictionary encoding is a good option. Could you please give a piece of advice - is it possible to embed decoding algorithm in Elastic? So that queries would be encoded and matches would decode using dictionaries? From where to start - any information is welcomed.

I encoded data and loaded into Elasticsearch. Satisfied with the result but next requirement/request is to embed encoding of queries and decoding of results. Is it possible?

Upvotes: 0

Views: 82

Answers (1)

Murat K.
Murat K.

Reputation: 952

If you want to manipulate the data before you ingest it into the Elasticsearch, I strongly recommend using Logstash, which is another leg of elk (elasticsearch, logstash, kibana).

There are 3 sections in it.

Input -> Decide which way and data type you want to ingest.

Filter -> Manipulate the data

Output -> Create index, set index lifecycle policy etc.

All of three have different plugins.

Input Plugins,

Filter Plugins,

Output Plugins

Upvotes: 0

Related Questions