CODEWITHSUNDEEP
elasticsearchelasticsearch-painless
Dennis
Dennis

Reputation: 105

Change field values using Elasticsearch Painless

I want to check the length of each field under the object alert.

If it's longer than X, change it to value was truncated since it was too long

"_source" : {
          "random" : 123455,
          },
          "@timestamp" : "2021-10-15T21:55:12.938+0000",
          "application" : "app",
          "alert" : {
            "app_key" : "XYZ",
            "host" : "Hostname",
            "check" : "CPU overloaded",
            "status" : "ok"
          },

For some reason, I get this error when trying to apply my script:

    "type" : "script_exception",
    "reason" : "runtime error",
    "script_stack" : [
      "java.base/java.util.HashMap$HashIterator.nextNode(HashMap.java:1584)",
      "java.base/java.util.HashMap$EntryIterator.next(HashMap.java:1617)",
      "java.base/java.util.HashMap$EntryIterator.next(HashMap.java:1615)",
      "for (alert in ctx._source.alert.entrySet())\n      {\n      ",
      "                               ^---- HERE"
    ],
    "script" : " ...",
    "lang" : "painless",
    "position" : {
      "offset" : 38,
      "start" : 7,
      "end" : 65
    },
    "caused_by" : {
      "type" : "concurrent_modification_exception",
      "reason" : null

Script:

{
  "script": {
    "lang": "painless",
    "source": """
      for (alert in ctx._source.alert.entrySet())
      {
      if (alert.getValue().length() > params.number_of_characters) {
        ctx._source.alert[alert] = "value was truncated since it was too long"
      }
      }
    """,
    "params": {
      "number_of_characters": 5
    }
  }
}

Upvotes: 2

Views: 2608

Answers (2)

Dennis
Dennis

Reputation: 105

Eventually, with the help of Nicolas, I used this:

PUT _ingest/pipeline/rawpayload-ingest/
{
  "processors": [
    {
      "script": {
        "description": "Loop through all fields in alert object and shorten them if needed",
        "lang": "painless",
        "source": """
          for (field in ctx.alert.entrySet()) {
            if (field.getValue() instanceof String) {
              if (field.getValue().length() > params.number_of_characters) {
              field.setValue(field.getValue().substring(0, params.number_of_characters) + "...truncated")
              }   
            }
          }
        """,
        "params": {
          "number_of_characters": 512
        }
      }
    }
  ]
}

Upvotes: 1

Nicolas Galler
Nicolas Galler

Reputation: 1309

You cannot modify the map directly while iterating over the set returned by entrySet, only through the setValue method of the entry.

Per the doc for entrySet():

Returns a Set view of the mappings contained in this map. The set is backed by the map, so changes to the map are reflected in the set, and vice-versa. If the map is modified while an iteration over the set is in progress (except through the iterator's own remove operation, or through the setValue operation on a map entry returned by the iterator) the results of the iteration are undefined. The set supports element removal, which removes the corresponding mapping from the map, via the Iterator.remove, Set.remove, removeAll, retainAll and clear operations. It does not support the add or addAll operations.

Try with the following script:

for (alert in ctx._source.alert.entrySet()){
    if (alert.getValue().length() > params.number_of_characters) {
        alert.setValue("value was truncated")      
    }      
}

Upvotes: 2

Related Questions