Fotis E.
Fotis E.

Reputation: 23

Logstash issue with config regex

I have the following log entry:

2017-08-29 01:10:11.111 [http-noo-111-exe-1] TRACE com.javasystemsolutions.xml.gateway.Actions - The XML Gateway encountered an error. The message was Server with id OPA is not configured.

The template in use was TEST_Create_Incident_elkmonitoring.

The server in use was OPA.

The input XML was <incident>
       <summary>Test Monitoring - Summary</summary>
       <notes>Test Monitoring - Summary</notes>
       <product>ELK FAQ</product> </incident> com.javasystemsolutions.xml.gateway.ServerNotFoundException: Server with id OPA is not configured
       at com.javasystemsolutions.xml.gateway.input.PostActions.doPost(PostActions.java:215) [jss-xmlgateway.jar:?]
       at com.javasystemsolutions.xml.gateway.input.PostActions.postAction(PostActions.java:86) [jss-xmlgateway.jar:?]

What I 'm trying to do, is to use regex and identify the text between the incident tags, but as it seems something is wrong although my regular expression works on regex101 website and the configtest returns Configuration OK. My config is the one below, does someone have an idea of what is wrong?

# The # character at the beginning of a line indicates a comment. Use
# comments to describe your configuration.
input {
    file {
        type => "logs"
        path => "C:/logs/*.log"
        add_field => [ "Application", "ELK_GW_Test" ]
        add_field => [ "secret", "1234" ]
        start_position => beginning

        codec => multiline {
            pattern => "(^%{TIMESTAMP_ISO8601})"
            #negate => true
            what => "previous"
        }
    }
}
filter {
    #multiline {
      #pattern => "(^%{TIMESTAMP_ISO8601})"
      #negate => true
      #what => "previous"
    #}
    #if "_grokparsefailure" in [tags] {
      #drop { }
    #}
    if [host] == "host1" {
        grok {
            match => { "message" => "%{TIMESTAMP_ISO8601:timestamp} %{NOTSPACE} %{LOGLEVEL:Severity} %{GREEDYDATA:log_message}"}
        }
        grok {
        match => {"message" => "<incident>(?<incident>[\s\S]*)</incident>"}
    }
    }
}
output {
    tcp {
        host => "host1.com"
        port => 1234
        codec => "json_lines"
    }
    #if  "The message was Server with id " in [log_message]  {
    #email {
            #from => "[email protected]"
            #subject => "Central logstash alert"
            #to => "[email protected]"
            #via => "smtp"
            #body => "The incident details are: %{incident} \nLog file: %{path}"
            #options => {
                #starttls => "true"
                #smtpIporHost => "email.XYZ.com"
                #port => "587"
                #userName => "[email protected]" 
                # email-server-mail-id
                # password => "password"
                #authenticationType => "LOGIN"
            #}
        #}
    #}
}

Upvotes: 0

Views: 160

Answers (1)

baudsp
baudsp

Reputation: 4100

This part of the configuration is wrong:

    grok {
        match => ["requested_incident", "(?s)<incident>.+?</incident>"]
    }

Try this instead:

    grok {
        match => {"message" => "<incident>(?<incident>[\s\S]*)</incident>"}
    }

I've used a custom pattern, which will search in the message field. What is found will go in a field called incident.

Upvotes: 1

Related Questions