CODEWITHSUNDEEP
elasticsearchlogstashkibanaelastic-stack
jit
jit

Reputation: 444

Using numerics as type in Elasticsearch

I am going to store transaction logs on elasticsearch. I am new to ELK stack and not sure about how I should implement this on ELK stack. My transaction is printing lines of log sequentially(upserts) and instead of logging these to a file I want to store these on ElastichSearch and later I will query the logs by the transactionId I have created.

Normally the URI for querying will be

/bookstore/books/_search

but in my case it must be like

/transactions/transactionId/_search

because I dont want to store lines as array attached to a single transaction record but I am not sure if this is a good practice to create a new type in the beginning of every transaction. I am not even sure if this is possible.

Can you give advices about storing these transaction data on elasticsearch?

Upvotes: 1

Views: 84

Answers (1)

soumitra goswami
soumitra goswami

Reputation: 891

if you want to query with a URI like /transactions/transactionId/_search, that means you are planning to create multiple types every time a new transactionid comes. Now , apart from this being a bad design, its not even possible to have more than one type in an index(post version 5.X I guess) and types have been completely removed since version 7.X . One work-around is if you use the transactionId itself as the document ID while creation. Then you can get the log associated with one transactionId by querying GET transactions/transactionId (read about the length restrictions of the document id though) but this might cause another issue, that being , there can be multiple logs for the same transaction, so each log entry having the same id would simply overwrite the previous entry. The best solution here will be to change how you query those records.

For this you can put transactionId as one of the fields in the json body, along with maybe a created time stamp at the time of insertion ( let ES create the documents with the auto generated id) and then query all logs associated with a transaction like :

POST transactions/_search
{
   "sort": [
    {
      "createdDate": {
        "order": "asc"
      }
    }
  ],
   "query":{
    "bool":{
      "must":[
        {
          "term":{
            "transactionId.keyword":"<transaction id>"
          }
        }
        ]
    }
  }
}

Hope, this helps

Upvotes: 1

Related Questions