CODEWITHSUNDEEP
sortingelasticsearchfiltering
Ilia Shakitko
Ilia Shakitko

Reputation: 1457

ElasticSearch filter impact on the sorting

I have a strange behavior in Elastic Search. Before I start using filter I had a query:

Query without FILTER

{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "_facility": {
              "query": "error",
              "operator": "AND"
            }
          }
        },
        {
          "match": {
            "_application": {
              "query": "live",
              "operator": "AND"
            }
          }
        }
      ],
      "must_not": [],
      "should": []
    }
  },
  "from": 0,
  "size": 10,
  "sort": [
    {
      "created_at": {
        "sort_mode": null,
        "order": "desc",
        "missing": null,
        "ignore_unmapped": null
      }
    },
    {
      "_score": {
        "sort_mode": null,
        "order": null,
        "missing": null,
        "ignore_unmapped": null
      }
    }
  ]
}

Then I had to add the FILTER

{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "_facility": {
              "query": "error",
              "operator": "AND"
            }
          }
        },
        {
          "match": {
            "_application": {
              "query": "live",
              "operator": "AND"
            }
          }
        }
      ],
      "must_not": [],
      "should": []
    }
  },
  "filter": {
    "and": {
      "filters": [
        {
          "range": {
            "created_at": {
              "from": 1373320800,
              "to": 1373493599,
              "include_lower": true,
              "include_upper": true
            },
            "_cache": true
          }
        }
      ]
    },
    "_cache": false
  },
  "from": 0,
  "size": 10,
  "sort": [
    {
      "created_at": {
        "sort_mode": null,
        "order": "desc",
        "missing": null,
        "ignore_unmapped": null
      }
    },
    {
      "_score": {
        "sort_mode": null,
        "order": null,
        "missing": null,
        "ignore_unmapped": null
      }
    }
  ]
}

and I have next issues:

1) results are not sorted by created_at correctly, looks like shuffled data

2) size - doesn't consider any custom values different from 10 (let say I want to display 20 [or 5] records, but I have 10)

Thanks for helping. Probably I am missing something in Elastic Search concept.

Upvotes: 0

Views: 2940

Answers (1)

imotov
imotov

Reputation: 30163

The problem is that "_cache": false in the and filter is actually outside of the filter. It should be one level deeper:

"filter": {
  "and": {
    "filters": [{
      "range": {
        "created_at": {
          "from": 1373320800,
          "to": 1373493599,
          "include_lower": true,
          "include_upper": true
        },
        "_cache": true
      }
    }],
    "_cache": false
  }
}

It throws off the Elasticsearch parser and it starts ignoring the rest of the query. By the way, these _cache statement are useless, because they just ensure default settings anyway. And as @Damien said, entire request would be much better as filtered query:

{
  "query": {
    "filtered": {
      "query": {
        "bool": {
          "must": [{
            "match": {
              "_facility": {
                "query": "error",
                "operator": "AND"
              }
            }
          }, {
            "match": {
              "_application": {
                "query": "live",
                "operator": "AND"
              }
            }
          }]
        }
      },
      "filter": {
        "and": {
          "filters": [{
            "range": {
              "created_at": {
                "from": 1373320800,
                "to": 1373493599,
                "include_lower": true,
                "include_upper": true
              },
              "_cache": true
            }
          }, {
            "match_all": {}
          }],
          "_cache": false
        }
      }
    }

  },
  "from": 0,
  "size": 10,
  "sort": [{
    "created_at": {
      "order": "desc"
    }
  }, {
    "_score": {}
  }]
}

Upvotes: 1

Related Questions