CODEWITHSUNDEEP
elasticsearchlogstashelastic-stack
schoel
schoel

Reputation: 863

Elasticsearch timestamp range doesn't work

I have Logstash as only input with the following filter and output:

filter {
  grok {
    match => { "message" => "^%{TIMESTAMP_ISO8601:timestamp}%{GREEDYDATA:message}$" }
  }

  date {
    match => [ "timestamp", ISO8601 ]
  }
}

output {
  elasticsearch {
    index => "logstash-%{[@metadata][beat]}"
    hosts => [ "${ES_HOSTS}" ]
    user => "${ES_USER}"
    password => "${ES_PASSWORD}"
    cacert => '/etc/logstash/certificates/ca.crt'
  }
}

When I try to do a range query over the timestamp it doesn't matter what I do, I always get the same result (that is not within the range specified). Example query in ruby code:

params = {
  query: {
    range: {
        "timestamp": {
          gte: "2022-09-22 17:00:00",
          lte: "now"
        }
    }
  },
}

response = ApiClient["_search"].post(
  params.to_json, content_type: 'application/json', accept: 'application/json'
)

Response, no matter what I pick in "gte" is (printing [hits][hits][_source][timestamp]):

2022-09-22 13:21:53,759
2022-09-22 13:21:53,762
2022-09-22 13:21:53,762
2022-09-22 13:21:53,773
2022-09-22 13:22:08,510
2022-09-22 13:22:51,219
2022-09-22 13:22:51,239
2022-09-22 13:22:51,247
2022-09-22 13:22:51,267
2022-09-22 13:23:04,325

I am able to get more / different results if I add size/from in the params. Very new to the entire ELK stack so it is probably something obvious, and I appreciate any help.

Upvotes: 0

Views: 657

Answers (1)

Fares
Fares

Reputation: 650

I have two leads from what I see of your query, the first being to use a timestamp in iso8601 format in your query, as this is the format of the timestamp field.

            "gte":"2022-09-22T17:00:00",
            "lt":"2022-09-24T12:00:00"

The second is to use "@timestamp" instead of "timestamp", with the @ character.

Upvotes: 2

Related Questions