How to call stored painless script function in elastisearch

Question

I am trying to use an example from

https://www.elastic.co/guide/en/elasticsearch/reference/6.4/modules-scripting-using.html

I have created a function and saved it.

POST http://localhost:9200/_scripts/calculate-score
{
  "script": {
    "lang": "painless",
    "source": "ctx._source.added + params.my_modifier"
  }
}

Try to call saved function

POST http://localhost:9200/users/user/_search
{
  "query": {
    "script": {
      "script": {
        "id": "calculate-score",
        "params": {
          "my_modifier": 2
        }
      }
    }
  }
}

And it returns an error: Variable [ctx] is not defined. I tried to use doc['added'] but received the same error. Please help me understand how to call the function.

Answer 1

You should try using doc['added'].value, let me explain you why and how. In short, because painless scripting language is rather simple but obscure.

Why can't ES find ctx variable?

The reason it cannot find ctx variable is because this painless script runs in "filter context" and such variable is not available in filter context. (If you are curious, there were 18 types of painless context as of ES 6.4).

In filter context there are only two variables available:

params (Map, read-only)

User-defined parameters passed in as part of the query.

doc (Map, read-only)

Contains the fields of the current document where each field is a List of values.

It should be enough to use doc['added'].value in your case:

POST /_scripts/calculate-score
{
  "script": {
    "lang": "painless",
    "source": "doc['added'].value + params.my_modifier"
  }
}

Should, because there will be another problem if we try to execute it (exactly like you did):

      "type": "script_exception",
      "reason": "runtime error",
      "script_stack": [
        "doc['added'].value + params.my_modifier",
        "^---- HERE"
      ],
      "script": "calculate-score",
      "lang": "painless",
      "caused_by": {
        "type": "class_cast_exception",
        "reason": "cannot cast def [long] to boolean"
      }

Because of its context, this script is expected to return a boolean:

Return

boolean

Return true if the current document should be returned as a result of the query, and false otherwise.

At this point we can understand why the script you were trying to execute did not make much sense for Elasticsearch: it is supposed to tell if a document matches a script query or not. If a script returns an integer, Elasticsearch wouldn't know if it is true or false.

How to make a stored script work in filter context?

As an example we can use the following script:

POST /_scripts/calculate-score1
{
  "script": {
    "lang": "painless",
    "source": "doc['added'].value > params.my_modifier"
  }
}

Now we can access the script:

POST /users/user/_search
{
  "query": {
    "script": {
      "script": {
        "id": "calculate-score1",
        "params": {
          "my_modifier": 2
        }
      }
    }
  }
}

And it will return all documents where added is greater than 2:

"hits": [
  {
    "_index": "users",
    "_type": "user",
    "_id": "1",
    "_score": 1,
    "_source": {
      "name": "John Doe",
      "added": 40
    }
  }
]

This time the script returned a boolean and Elasticsearch managed to use it.

If you are curious, range query can do the same job, without scripting.

Why do I have to put .value after doc['added']?

If you try to access doc['added'] directly you may notice that the error message is different:

POST /_scripts/calculate-score
{
  "script": {
    "lang": "painless",
    "source": "doc['added'] + params.my_modifier"
  }
}

      "type": "script_exception",
      "reason": "runtime error",
      "script_stack": [
        "doc['added'] + params.my_modifier",
        "                     ^---- HERE"
      ],
      "script": "calculate-score",
      "lang": "painless",
      "caused_by": {
        "type": "class_cast_exception",
        "reason": "Cannot apply [+] operation to types [org.elasticsearch.index.fielddata.ScriptDocValues.Longs] and [java.lang.Integer]."
      }

Once again painless shows us its obscurity: when accessing the field 'added' of the document, we obtain an instance of org.elasticsearch.index.fielddata.ScriptDocValues.Longs, which Java Virtual Machine denies to add to an integer (we can't blame Java here).

So we have to actually call .getValue() method, which, translated in painless, is simply .value.

What if I want to change that field in a document?

What if you want to add 2 to field added of some document, and save the updated document? Update API can do this.

It operates in update context, which actually has got ctx variable defined, which in turn has access to the original JSON document via ctx['_source'].

We might create a new script:

POST /_scripts/add-some
{
  "script": {
    "lang": "painless",
    "source": "ctx['_source']['added'] += params.my_modifier"
  }
}

Now we can use it:

POST /users/user/1/_update
{
    "script" : {
        "id": "add-some",
        "params" : {
            "my_modifier" : 2
        }
    }
}

Why the example from the documentation doesn't work?

Apparently, because it is wrong. This script (from this documentation page):

POST _scripts/calculate-score
{
  "script": {
    "lang": "painless",
    "source": "Math.log(_score * 2) + params.my_modifier"
  }
}

is later executed in filter context (in a search request, in a script query), and, as we now know, there is no _score variable available.

This script would kind of make sense only in score context, when running a funtion_score query which allows to twiggle the relevance score of the documents.

Final note

I would like to mention that in general, it's recommended to avoid using scripts because their performance is poor.