CODEWITHSUNDEEP
groovyelasticsearch
Smair
Smair

Reputation: 353

Elasticsearch: transform date with groovy script

i have the following (simplified) mapping:

{
  "event_date": {
    "_source": { "enabled": true },
    "_all": { "enabled": true },
    "dynamic": "strict",
    "properties": {
      "start_date_time": {
        "type": "date",
        "format": "dateOptionalTime"
      },
      "start_date_day": {
        "type": "date",
        "format": "dateOptionalTime",
        "index": "not_analyzed"
      }
    }
  }
}

The indexed objects will look like this:

{
  "start_date_time": "2017-05-08T18:23:45+0200"
}

The property start_date_day should always contain the same date, but with the time set to 00:00:00. In the example above start_date_day must be "2017-05-08T00:00:00+0200".

I think,it is possible to achieve this with a transform mapping and a groovy script, but my developed groovy code did not work in the elasticsearch-context and I am not that familiar with the groovy language.

Maybe someone has an idea on how to solve this?

Upvotes: 2

Views: 2990

Answers (3)

Mateusz Dymczyk
Mateusz Dymczyk

Reputation: 15141

Yes this is doable, for testing/running you might need to turn on script.groovy.sandbox.enabled: true in ../conf/elasticsearch.yml first.

PUT datetest/
{
    "mappings": {
        "event_date": {
            "_source": { "enabled": true },
            "_all": { "enabled": true },
            "dynamic": "strict",
            "transform" : {
                "script" : "ctx._source['start_date_day'] = new Date().parse(\"yyyy-MM-dd\", ctx._source['start_date_time']).format(\"yyyy-MM-dd\");",
                "lang": "groovy"
            },
            "properties": {
                "start_date_time": {
                    "type": "date",
                    "format": "dateOptionalTime"
                },
                "start_date_day": {
                    "type": "date",
                    "format": "dateOptionalTime",
                    "index": "not_analyzed",
                    "store": "yes"
                }
            }
        }
    }
}

Sample data:

PUT /datetest/event_date/1
{
  "start_date_time": "2017-05-08T18:23:45+0200"
}

Sample output:

GET /datetest/event_date/_search
{
  "query": {
    "match_all": {}
  },
  "fields": ["start_date_time","start_date_day"]
}

{
   "took": 2,
   "timed_out": false,
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "hits": {
      "total": 1,
      "max_score": 1,
      "hits": [
         {
            "_index": "datetest",
            "_type": "event_date",
            "_id": "1",
            "_score": 1,
            "fields": {
               "start_date_day": [
                  "2017-05-08T00:00:00.000Z"
               ],
               "start_date_time": [
                  "2017-05-08T18:23:45+0200"
               ]
            }
         }
      ]
   }
}

Upvotes: 3

Val
Val

Reputation: 217474

Based on your comments, since you don't need the hour part, you can simply define the mapping for the start_date_day field using the date format and use the following transform like this:

{
  "event_date": {
    "_source": {
      "enabled": true
    },
    "_all": {
      "enabled": true
    },
    "dynamic": "strict",
    "transform": {
      "script": "ctx._source['start_date_day'] = ctx._source['start_date_time'].split('T')[0]",
      "lang": "groovy"
    },
    "properties": {
      "start_date_time": {
        "type": "date",
        "format": "dateOptionalTime"
      },
      "start_date_day": {
        "type": "date",
        "format": "date",
        "index": "not_analyzed"
      }
    }
  }
}

ES will only store the date part and leave the hours and timezone out.

However, you should note that when using transform, the original source is stored without modification, the result of the transform will be indexed though, and hence searchable.

Upvotes: 1

Pred05
Pred05

Reputation: 542

i think it is the format which isn't good. "date" seems to be MM/DD/YYYY and only this. If you want the time you need datetime format. I have found this link which can help you : Elastic date format

You can try to change type "date" to "basic_date_time".

Upvotes: 1

Related Questions