CODEWITHSUNDEEP
elasticsearchelasticsearch-painless
brianNotBob
brianNotBob

Reputation: 596

Is it possible to access field outside of _source in elasticsearch Ingest Pipeline

I am having problems creating an ingest-pipe in elasticstack that has access to fields outside of _source object. I am using elasticstack 7.8. Here is a trivial example:

POST _ingest/pipeline/_simulate
{
  "pipeline": {
    "processors": [
      {
         "set": {
            "field": "foo_found",
            "value": "true",
            "if": "ctx.foo != null && ctx.foo =~ /foo/"
         }
      },
      {
         "set": {
            "field": "bar_found",
            "value": "true",
            "if": "bar != null && bar =~ /bar/"
         }
      }
    ]
  },
  "docs": [
    {
      "_source": {
        "foo": "foofoo"
      },
      "bar":"barbar"
    }
  ]
}

The desired result is below:

{
 "docs": [
   {
     "doc": {
      "_source": {
        "foo_found": "true",
        "foo": "foofoo",
        "bar_found": "true"
      },
      "bar":"barbar"
     }
   }
 ]
}

The pipeline fails to run.

Is it possible to access the bar field which is outside of _source object?

The example will run if the second set processor is removed as in the following:

POST _ingest/pipeline/_simulate
{
  "pipeline": {
    "processors": [
      {
         "set": {
            "field": "foo_found",
            "value": "true",
            "if": "ctx.foo != null && ctx.foo =~ /foo/"
         }
      }
    ]
  },
  "docs": [
    {
      "_source": {
        "foo": "foofoo"
      },
      "bar":"barbar"
    }
  ]
}

Upvotes: 0

Views: 268

Answers (1)

florianvazelle
florianvazelle

Reputation: 313

According to the Elasticsearch documentation, to simulate an ingest pipeline we can only pass _id, _index, _routing and/or _source to the sample documents.

Upvotes: 0

Related Questions