Elasticsearch function_score decay not working, always returns 1

Question

I've been trying to fix this for hours, but nothing seems to change the return value of the function_score decay function. It's simply 1 at all time. It looks like it can't get the integer of the field I'm specifying?

The data model looks like this (obviously fake):

{
  "basics": {
    "name": "Mr Augustus Flybynight (Jim)",
    "name_pref": "Jim",
    "location": {
      "city": "Melbourne",
      "postalCode": "3040",
      "meta": {
        "country": "Australia"
      },
      "region": "VIC",
      "address": "iytiytiyt, tyiuyti"
    },
    "email": "augustus.flybynight2@gmail.com",
    "applicantNumber": "11882",
    "name_first": "Augustus",
    "meta": {
      "alternateContact": "",
      "lastModified": 1473353751,
      "alternateName": "",
      "notificationType": "-1",
      "alternatePhones": [

      ],
      "gender": "M"
    },
    "name_last": "Flybynight",
    "phone": "44556677"
  }
}

I have 3 duplicates of this entity which the only difference is their timestamp (basics.meta.lastModified). I'm trying to create a 'closer is better' functional score so that the latest comes to the top. We haven't mapped the timestamp as a date yet, but it is mapped as an integer.

When trying to query with the following

{
  "query": {
    "function_score": {
      "functions": [
        {
          "gauss": {
            "basics.meta.lastModified": {
              "origin": 1474868635, // now
              "offset": 86400, // one day
              "scale": 604800, // seven days
              "decay": 0.5
            }
          },
          "weight": 2
        }
      ],
      "query": {
        "bool": {
          "should": [
            {
              "match": {
                "_all": "augustus flybynight"
              }
            },
            {
              "match": {
                "basics.all_names.all_names_identifier_whitespace": {
                  "query": "augustus flybynight",
                  "boost": 2
                }
              }
            },
            {
              "match": {
                "basics.email.email_identifier_keyword": {
                  "query": "augustus flybynight",
                  "boost": 3
                }
              }
            },
            {
              "match": {
                "basics.applicantNumber.applicantNumber_identifier_keyword": {
                  "query": "augustus flybynight",
                  "boost": 3
                }
              }
            },
            {
              "wildcard": {
                "basics.email.email_identifier_keyword": {
                  "wildcard": "augustus flybynight*",
                  "boost": 2
                }
              }
            },
            {
              "wildcard": {
                "basics.all_names.all_names_identifier_whitespace": {
                  "wildcard": "augustus flybynight*"
                }
              }
            }
          ],
          "must": []
        }
      }
    }
  },
  "size": 25,
  "from": 0,
  "min_score": 0.2
}

But this always returns '1' for the functional score, which is then multiplied to the query and doesn't affect it. It's the weirdest thing.

When looking at the explanation, this is what's returned:

{
  "value": 1,
  "description": "min of:",
  "details": [
    {
      "value": 1,
      "description": "product of:",
      "details": [
        {
          "value": 1,
          "description": "Function for field basics.meta.lastModified:",
          "details": [
            {
              "value": 1,
              "description": "max(0.0, ((2.0 - MIN[0.0])/2.0)",
              "details": [

              ]
            }
          ]
        },
        {
          "value": 1,
          "description": "weight",
          "details": [

          ]
        }
      ]
    },
    {
      "value": 3.4028235e+38,
      "description": "maxBoost",
      "details": [

      ]
    }
  ]
}

Seems like 'MIN[0.0]' is the part that should be returning the timestamp, but it isn't, instead returning 0 and making the decay function always 1. if I make the decay parameters stricter, like origin:0, offset:0, scale:1 and decay:0.5, I'd expect the function_score to be close to 0, but it's still 1.

Please help. I've been trying everything and there doesn't seem to be a lot of examples online. Any suggestions would be welcomed.

Answer 1

For those hitting the same issue, I've finally found the culprit.

It seems that someone didn't setup the mappings properly at that the basics.meta property was set as a nested type, but since it wasn't populated as such (you'd think that would of caused an issue when indexing data?), when trying to access the data within it, it always returned MIN[0.0] because it simply couldn't find the value of the property.

So yeah, if you ever hit this issue, thoroughly look through your mappings instead of wasting a whole day like I did :|