CODEWITHSUNDEEP
parsingdynamiclogstashgrok
Gintautas Sulskus
Gintautas Sulskus

Reputation: 25

Is it possible to parse dynamic xml-structured log contents with Grok?

Is it feasible using Grok to parse dynamic xml-structured log contents, such as:

<tag_1> contents </tag_1> ... <tag_N> contents </tag_N>

where "tag_*" would be the field name and "contents" - the actual contents. Therefore the parsed message would look like:

{
  "tag_1": [
    [
      "contents"
    ]
  ],

....

  "tag_N": [
    [
      "contents"
    ]
  ]
}

Upvotes: 1

Views: 689

Answers (1)

Alcanzar
Alcanzar

Reputation: 17165

Not with grok. You will need to resort to ruby code to parse the XML and toss it into the event structure.

If your XML is super regular (ie has a root element and only one level under it), you could maybe use code like this:

filter {
    ruby {
        code => "
        msg = event['message'].split('><');
        for part in msg
            endpos = part.index('</')
            startpos = part.index('>')
            if !endpos.nil? && !startpos.nil? then
                tag = part[0,startpos];
                text = part[startpos+1,endpos-startpos-1];
                event[tag]=text
            end
        end
        "
    }
}

If your xml is more complex, you are going to have to resort to a real XML parser and figure out how to use it with logstash (I've never brought an external library into logstash).

Upvotes: 1

Related Questions