Elastic search filter not working for multi value json

Question

Hi My elastic search index has the mapping as.

"userId": {
      "properties": {
        "name": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "userId": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "userName": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        }
      }
    }

Also my search query looks like this GET : http://localhost:5000/questions/_search

Body is

{
  "query": {
    "bool": {
      "filter": [
        { "term": { "userId.userId": "testuser@demo.com"
        }}
   
      ]
    }
  }
}

I am always getting 0 hits. Is there a better value to query multivalue json.

Answer 1

userId.userId field is of text type. If no analyzer is defined, elasticsearch by default uses a standard analyzer. This will tokenize testuser@demo.com into

{
  "tokens": [
    {
      "token": "testuser",
      "start_offset": 0,
      "end_offset": 8,
      "type": "<ALPHANUM>",
      "position": 0
    },
    {
      "token": "demo.com",
      "start_offset": 9,
      "end_offset": 17,
      "type": "<ALPHANUM>",
      "position": 1
    }
  ]
}

You need to use "userId.userId.keyword" field on the userId.userId field. This uses the keyword analyzer instead of the standard analyzer (notice the ".keyword" after userId.userId field).

You are getting 0 hits, because the term query, always searches for exact matching term. And as you are using the standard analyzer (which is the default one) for searching, you will not get correct results

{
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "userId.userId.keyword": "testuser@demo.com"
          }
        }
      ]
    }
  }
}

If you want to search for multiple fields use the terms query

{
  "query": {
    "bool": {
      "filter": [
        {
          "terms": {
            "userId.userId.keyword": [
              "testuser@demo.com",
              "abc.com"
            ]
          }
        }
      ]
    }
  }
}

Update 1:

You can use the must_not clause along with the term query to get all records that have userId not equal to testuser@demo.com

{
  "query": {
    "bool": {
      "must_not": {
        "term": {
          "userId.userId.keyword": "testuser@demo.com"
        }
      }
    }
  }
}

Answer 2

Terms query returns documents that contain one or more exact terms in a provided field.The terms query is the same as the term query, except you can search for multiple values.

{
  "query": {
    "terms": {
      "userId.userId": [ "testuser@demo.com", "other@demo.com" ],
      "boost": 1.0
    }
  }
}