CODEWITHSUNDEEP
elasticsearch
Rajneesh
Rajneesh

Reputation: 51

Elastic search ---: MUST_NOT query not working

I have a query in which i want to add a must_not clause that would discard all records that have blank data for a some field. I tried a lot of ways but none worked. when I issue the same query (mentioned below) with other specific fields then it works fine.

this query should get all records that do not have "registrationType1" field empty/blank

query:
{
"size": 20,
"_source": [
"registrationType1"
],
"query": {
"bool": {
"must_not": [
{
"term": {
"registrationType1": ""
}
}
]
}
}
}

the results below still contains "registrationType1" with empty values

results:

**"_source": {
"registrationType1": ""}}
, * {
"_index": "oh_animal",
"_type": "animals",
"_id": "3842002",
"_score": 1,
"_source": {
"registrationType1": "A&R"}}
, * {
"_index": "oh_animal",
"_type": "animals",
"_id": "3842033",
"_score": 1,
"_source": {
"registrationType1": "AMHA"}}
, * {
"_index": "oh_animal",
"_type": "animals",
"_id": "3842213",
"_score": 1,
"_source": {
"registrationType1": "AMHA"}}
, * {
"_index": "oh_animal",
"_type": "animals",
"_id": "3842963",
"_score": 1,
"_source": {
"registrationType1": ""}}
, * {
"_index": "oh_animal",
"_type": "animals",
"_id": "3869063",
"_score": 1,
"_source": {
"registrationType1": ""}}**

PFB mappings for the field above

"registrationType1": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
}

Upvotes: 5

Views: 4976

Answers (4)

Rajneesh
Rajneesh

Reputation: 51

There was some issue with the mappings itself, I deleted the index and re-indexed it with new mappings and its working now.

Upvotes: 0

Shabbir
Shabbir

Reputation: 153

I am using elasticsearch version 7.2, I replicated your data and ingested in my elastic index,and tried querying with and without .keyword.

I am getting the desired result when using the ".keyword" in the field name.It is not returning the docs which have registrationType1="".

Note - The query does not works when not using the ".keyword"

I have added my sample code below, have a look if that helps.

from elasticsearch import Elasticsearch
es = Elasticsearch()
es.indices.create(index="test", ignore=400, body={
"mappings": {
    "_doc": {
        "properties": {
            "registrationType1": {
                "type": "text",
                "field": {
                    "keyword": {
                        "type": "keyword"
                    }
                }
            }
        }
    }
}
})

data = {
"registrationType1": ""
}

es.index(index="test",doc_type="_doc",body=data,id=1)
search = es.search(index="test", body={
"size": 20,
"_source": [
    "registrationType1"
],
"query": {
    "bool": {
    "must_not": [
        {
            "term": {
                "registrationType1.keyword": ""
            }
        }
    ]
    }
}
})

print(search)

Executing the above should not return any results as we are inserting empty for the field

Upvotes: 0

Kamal Kunjapur
Kamal Kunjapur

Reputation: 8860

If you do not specify any text value on the text fields, there is basically nothing to analyze and return the documents accordingly.

In similar way, if you remove must_not and replace it with must, it would show empty results.

What you can do is, looking at your mapping, query must_not on keyword field. Keyword fields won't be analysed and in that way your query would return the results as you expect.

Query

POST myemptyindex/_search
{
  "query": {
    "bool": {
      "must_not": [
        {
          "term": {
            "registrationType1.keyword": ""
          }
        }
      ]
    }
  }
}

Hope this helps!

Upvotes: 1

Val
Val

Reputation: 217554

You need to use the keyword subfield in order to do this:

{
  "size": 20,
  "_source": [
    "registrationType1"
  ],
  "query": {
    "bool": {
      "must_not": [
        {
          "term": {
            "registrationType1.keyword": ""       <-- change this
          }
        }
      ]
    }
  }
}

Upvotes: 3

Related Questions