CODEWITHSUNDEEP
elasticsearch
Miguel Barrios
Miguel Barrios

Reputation: 483

ELASTICSEARCH - Include date automatically without a predefined date field

It is possible to include a "date and time" field in a document that receives elasticsearch without it being previously defined. The date and time corresponds to the one received by the json to elasticsearch

This is the mapping:

{
  "mappings": {
    "properties": {
      "entries":{"type": "nested"
      }
    } 
  }
}

Is it possible that it can be defined in the mapping field so that elasticsearch includes the current date automatically?

Upvotes: 1

Views: 3395

Answers (2)

Val
Val

Reputation: 217314

What you can do is to define an ingest pipeline to automatically add a date field when your document are indexed.

First, create a pipeline, like this (_ingest.timestamp is a built-in field that you can access):

PUT _ingest/pipeline/add-current-time
{
  "description" : "automatically add the current time to the documents",
  "processors" : [
    {
      "set" : {
        "field": "@timestamp",
        "value": "{{{_ingest.timestamp}}}"
      }
    }
  ]
}

Then when you index a new document, you need to reference the pipeline, like this:

PUT test-index/_doc/1?pipeline=add-current-time
{
   "my_field": "test"
}

After indexing, the document would look like this:

GET test-index/_doc/1
=>

{
   "@timestamp": "2020-08-12T15:48:00.000Z",
   "my_field": "test"
}

UPDATE:

Since you're using index templates, it's even easier because you can define a default pipeline to be run for each indexed documents.

In your index templates, you need to add this to the index settings:

{
  "order": 1,
  "index_patterns": [
    "attom"
  ],
  "aliases": {},
  "settings": {
    "index": {
      "number_of_shards": "5",
      "number_of_replicas": "1",
      "default_pipeline": "add-current-time"     <--- add this
    }
  },
  ...

Then you can keep indexing documents without referencing the pipeline, it will be automatic.

Upvotes: 6

panda
panda

Reputation: 11

"value": "{{{_ingest.timestamp}}}"

Source

Upvotes: 0

Related Questions