CODEWITHSUNDEEP
elasticsearch
Sebastian Slutzky
Sebastian Slutzky

Reputation: 379

Can I force ES to return dates in epoch_millis format?

I have this field mapping

"time": {
              "type": "date",
              "format": "strict_date_optional_time||epoch_millis"
           },

and I'm querying documents with this filter:

"range": {
            "time": {
              "gt": "1473157500000",
              "lte": "1473158700000",
            "format": "epoch_millis"
            }

this works and returns documents, but the result show the time field in a different format: 

"time": "2016-09-06T10:25:23.678",

Is it possible to force queries to be returned in epoch_millis?

Upvotes: 1

Views: 2718

Answers (2)

Bitswazsky
Bitswazsky

Reputation: 4698

ES 6.5 onwards, we need to use docvalue_fields in this specific structure, as the fielddata_fields has been deprecated. E.g. let's say we ingested a json doc of the following format:

{
  "@version": "1",
  "@timestamp": "2019-01-29T10:01:19.217Z",
  "host": "C02NTJRMG3QD",
  "message": "hello world!"
}

Now let's execute the following get query with docvalue_fields:

curl -X GET \
  http://localhost:9200/myindex/_search \
  -H 'Content-Type: application/json' \
  -d '{
  "query": {
    "match_all": {}
  },
  "docvalue_fields": [
    {
      "field": "@timestamp",
      "format": "epoch_millis"
    }
  ]
}'

And, we'll get the following response: 

{
    "took": 15,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": 1,
        "max_score": 1,
        "hits": [
            {
                "_index": "myindex",
                "_type": "doc",
                "_id": "mY8OmWgBF3ZItz5TVDiL",
                "_score": 1,
                "_source": {
                    "@version": "1",
                    "@timestamp": "2019-01-29T10:01:19.217Z",
                    "host": "C02NTJRMG3QD",
                    "message": "hello world!"
                },
                "fields": {
                    "@timestamp": [
                        "1548756079217"
                    ]
                }
            }
        ]
    }
}

Upvotes: 1

keety
keety

Reputation: 17441

The _source always returns the data in the original document.
Ideally I feel it maybe more desirable and flexible to convert the _source data to the desired format for presentation or otherwise on the client end.
However for the above use case you could use fielddata_fields.

fielddata_fields would return fields in the format of how the field-data is actually stored which in case of date field happens to be epoch_millis.

From documentation:

Allows to return the field data representation of a field for each hit Field data fields can work on fields that are not stored. It’s important to understand that using the fielddata_fields parameter will cause the terms for that field to be loaded to memory (cached), which will result in more memory consumption.

Example:

post <index>/_search
{
    "fielddata_fields": [
       "time"
    ]    
}

Upvotes: 2

Related Questions