CODEWITHSUNDEEP
elasticsearch
Keith John Hutchison
Keith John Hutchison

Reputation: 5287

What are good ways to solve a strange data retrieval issue in elastic search?

I've got a strange issue with an elastic search server.

The elastic search version is 1.6. 'records' is the name of the type. The url for the search is http://some.domain:9200/user/records/_search. The field mapping for 'un' is string.

The following query which been working for years is sometimes failing depending on the value of {someId} newer ids fail, old ones work. The data is there it's just not being found ...

{
  "from": 0,
  "size": 1,
  "sort": {
    "un": "desc",
    "_score": "desc"
  },
  "query": {
    "query_string": {
      "query": "un:\"{someId}\"",
      "fields": [
        "id",
        "un",
        "e",
        "fn",
        "ln",
        "bn",
        "jt",
        "sy",
        "c",
        "st",
        "p",
        "fbid",
        "lnid"
      ]
    }
  }
}

After doing some diagnostics I discovered the following query always works whether or not {someId} is old or new ...

{
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "default_field": "records.un",
            "query": "{someId}"
          }
        }
      ],
      "must_not": [],
      "should": []
    }
  },
  "from": 0,
  "size": 10,
  "sort": [],
  "aggs": {}
}

This is a sample document that matches with the second query and fails with the first.

{
    "un": "xxxxxxx.xxxxxxx",
    "e": "xxxxxxx",
    "pswd": "xxxxxxx",
    "fn": "xxxxxxx",
    "ln": "xxxxxxx",
    "bn": "xxxxxxx",
    "jt": "",
    "sy": "xxxxxxx",
    "urole": "User",
    "id": "xxxxxxx",
    "status": "1",
    "lld": "201704280016",
    "cd": "201702100132",
    "md": "201704280549",
    "cc": "0",
    "p": "",
    "logo": "",
    "mlogo": "",
    "ad": "201702100132",
    "com": "xxxxxxx",
    "rr": "true",
    "sid": "00000000-0000-0000-0000-000000000000",
    "fbidp": "",
    "lnidp": "",
    "role": "Lots of data is in this one",
    "dim": "",
    "drm": "",
    "drcm": "xxxxxxx",
    "drcfbm": "xxxxxxx",
    "drclnm": "xxxxxxx",
    "as": "false",
    "apr": "true",
    "iuid": "xxxxxxx",
    "vcount": "9",
    "pplatform": "",
    "pname": "",
    "pid": "00000000-0000-0000-0000-000000000000",
    "preciept": "",
    "ms": "Free"
}

I'm thinking that reindexing the server might solve the issue. What are good ways to solve strange data retrieval issues in elastic search?

Upvotes: 0

Views: 97

Answers (1)

avr
avr

Reputation: 4893

There is significant difference between your first ("query": "un:\"{someId}\"") query and second ("query": "{someId}") query. In former query as you are wrapping someId in quotes as a result it will search for exact phrase i.e if you have xxx.yyy then it will look for whole id including dot(.) so id will be matched only when id doesn't contains dot where as in latter query your someId will be analyzed i.e xxx.yyy will be tokenized into two strings (xxx and yyy) and it will be matched if you have dot.

You need to change mappings of un field. If you are not doing any full-text search queries on un then I'd suggest you to make it not_analyzed. Otherwise you need to use different analyzer like whitespace instead of default standard analyzer. I'd really suggest to go with former solution as it(structured exact fields) is more efficient than latter.

Upvotes: 1

Related Questions