CODEWITHSUNDEEP
javaelasticsearch
b15
b15

Reputation: 2361

Using multiple filters with OR and multi-field

We have an index with a keyword field that is very often an ip address, but not always. We'd like to be able to search this index on that field using not just keywords but also CIDR notation, which is supported only for fields of type 'ip'. On the surface, this looks like a use case for multi-fields, so we have an index with mappings:

{
    "mappings": {
        "my_field": {
            "type": "keyword"
            "fields": {
                "ip": {
                    "type": "ip"
                    "ignore_malformed": true
                }  
            }
        }
    }
}

So, when our application has a set of non-ip addresses, ip addresses, and CIDR-notation blocks/ranges of ip addresses and needs to query by them, our application splits that set into one set with non-ip addresses and another with ip addresses/CIDR-notation blocks and makes two separate terms filters from them in my query, like so:

{
  "query": {
    "bool": {
      "filter": [
        {
          "terms": {
            "my_field.ip": [
              "123.123.123.0/24",
              "192.168.0.1",
              "192.168.16.255",
              "192.169.1.0/24"
            ]
          }
        },
        {
          "terms": {
            "my_field": [
              "someDomain.com",
              "notAnIp.net"
            ]
          }
        }
      ]
    }
  }
}

This returns no records. I assume because it's not treating the two filters as an OR. I want all of the records that match the first terms query PLUS all of the records that match the second terms query. What is the best way to achieve this? Hoping my application doesn't need to send two separate requests.

Upvotes: 1

Views: 128

Answers (2)

Amit
Amit

Reputation: 32386

You misunderstood the filter clause, it will filter the documents without should or must clause in your boolean query, please refer to boolean queries official doc for more info.

when you use should or must clause with a filter then its meaning is changed and it will start returning the documents, refer to @ESCoder answer on how to achieve it.

Also please note as you are using only the filter clause, the score will not be calculated and all the documents in your search results will have the same score as 0, which might not be something you want, if you want to score to happen then you need to use the query clause.

Upvotes: 1

Bhavya
Bhavya

Reputation: 16192

Try out this below query:

{
  "query": {
    "bool": {
      "should": [
        {
          "bool": {
            "filter": {
              "terms": {
                "my_field.ip": [
                  "123.123.123.0/24",
                  "192.168.0.1",
                  "192.168.16.255",
                  "192.169.1.0/24"
                ]
              }
            }
          }
        },
        {
          "bool": {
            "filter": {
              "terms": {
                "my_field": [
                  "someDomain.com",
                  "notAnIp.net"
                ]
              }
            }
          }
        }
      ],
      "minimum_should_match": 1
    }
  }
}

Upvotes: 1

Related Questions