CODEWITHSUNDEEP
elasticsearchkibana
Bolo
Bolo

Reputation: 93

ElasticSearch query returns wrong results

I'm relatively new to ElasticSearch and encountered this issue which I can't seem to get why.

So for this particular field, it seems to be treating all the values to be zero, even though the individual records are non-zero values. This only seems to happen to this number field and not other similar fields (such as cpu pct, mem pct etc)

The records only show when I query for records that have 'system.filesystem.used.pct == 0', whereas none of them show when I do something like 'system.filesystem.used.pct > 0'.

Kibana Discover Results

I also did the querying in the dev tools in kibana like so, yet I don't get any results:

GET metricbeat-*/_search{
  "query": {
    "range":{
      "system.filesystem.used.pct":{
         "gt":0
        }
       }
     }
}

enter image description here

However, if I did this, I will get all non-zero results, just like in discover:

GET metricbeat-*/_search
{
  "query": {
    "term": {
      "system.filesytem.used.pct":0
     }
  }
}

Sample match

Upvotes: 0

Views: 659

Answers (1)

Bhavya
Bhavya

Reputation: 16172

As pointed out by @Ron Serruya, there is a mapping issue. The mapping for system.filesytem.used.pct is detected as to be of integer type. Since, you are getting the expected search results for cpu.pct field, the mapping of cpu.pct, must have been of float type

CASE 1:

If you index the two sample data as (in the same order)

{
  "count": 0.45
}
{
  "count": 0
}

Then float data type is detected by elasticsearch (if you are using dynamic mapping). this is because the detection of the field type depends on the first data that you have inserted in the field.

CASE 2:

Now, if you index the data in this order

{
  "count": 0
}
{
  "count": 0.45
}

Here elasticsearch will detect count to be of long data type.

You need to recreate the index, with the new index mapping, reindex the data and then run the search query on system.filesytem.used.pct

Modified index mapping will be

{
  "mappings": {
    "properties": {
      "system": {
        "properties": {
          "filesytem": {
            "properties": {
              "used": {
                "properties": {
                  "pct": {
                    "type": "float"
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

Upvotes: 2

Related Questions