CODEWITHSUNDEEP
elasticsearch
edgard habanbou
edgard habanbou

Reputation: 23

issue in creating a pipeline in elasticSearch

I'm trying to ingest a pipeline that contains grok, date and remove processors but i am getting a missing field error despite explicitly mentioning the field "message" under the docs

GET _ingest/pipeline/_simulate
    {
      "pipeline" : {
        "processors" : [
          {
            "grok" : {
              "field" : "message", 
                "pattern" : "%{COMMONAPACHELOG}"
            }
          },
          {
            "date" : {
              "match_field" : "timestamp", 
              "match_formats" : ["dd/MMM/YYYY:HH:mm:ss Z"]
              
            }
          }, 
          {
          "remove" : {
            "field" :  "message" 
          }
      }
    ]
  }, 

    "docs" : [
    {
      "_source" :  {
      "message" : "52.35.38.35 -- [19/Apr/2016:12:00:04 +0200] \"GET/ HTTP/1.1\" 200 24"
      }, 
      "_index" :  "indexer" 
    }
  ]
  }

and i'm Getting this Error please help

{
  "error" : {
    "root_cause" : [
      {
        "type" : "parse_exception",
        "reason" : "[patterns] required property is missing",
        "property_name" : "patterns",
        "processor_type" : "grok",
        "suppressed" : [
          {
            "type" : "parse_exception",
            "reason" : "[field] required property is missing",
            "property_name" : "field",
            "processor_type" : "date"
          }
        ]
      }
    ],
    "type" : "parse_exception",
    "reason" : "[patterns] required property is missing",
    "property_name" : "patterns",
    "processor_type" : "grok",
    "suppressed" : [
      {
        "type" : "parse_exception",
        "reason" : "[field] required property is missing",
        "property_name" : "field",
        "processor_type" : "date"
      }
    ]
  },
  "status" : 400
}

i tried to look for a video on youtube and i found someone with the same code and it executed well here's the video https://www.youtube.com/watch?v=PEHnBa19Gxs&t=1s it's on minute 34

Upvotes: 0

Views: 370

Answers (1)

edgard habanbou
edgard habanbou

Reputation: 23

as it turns out that it worked at the youtube guy because it was on an older version. this will work on the newer version

GET _ingest/pipeline/_simulate
    {
      "pipeline" : {
        "processors" : [
          {
            "grok" : {
              
              "field" : "message", 
              "patterns" : ["%{COMMONAPACHELOG}"]
            }
          },
          {
            "date" : {
              "field" : "timestamp", 
              "formats" : ["dd/MMM/YYYY:HH:mm:ss Z"]
            }
          }, 
          {
          "remove" : {
            "field" :  "message" 
          }
      }
    ]
  }, 

    "docs" : [
    {
      "_source" :  {
      "message" : "52.35.38.35 - - [19/Apr/2016:12:00:04 +0200] \"GET/ HTTP/1.1\" 200 24"
      }, 
      "_index" :  "indexer" 
    }
  ]
  }

Upvotes: 0

Related Questions