CODEWITHSUNDEEP
elasticsearchlogstashlogstash-grok
yuliansen
yuliansen

Reputation: 520

grok not parsing logs

Log Sample


[2020-01-09 04:45:56] VERBOSE[20735][C-0000ccf3] pbx.c: Executing [9081228577525@from-internal:9] Macro("PJSIP/3512-00010e39", "dialout-trunk,1,081228577525,,off") in new stack

I'm trying to parse some logs, I have tested some logs I have made on and it returning the result I need. But when I combining it with my config and run it, the logs not parsed into the index.

here is my config:

input{
        beats{
                port=>5044
        }
}
filter
{
        if [type]=="asterisk_debug"
        {
                if [message] =~ /^\[/
                {
                        grok
                        {
                                match =>
                                {
                                        "message" => "\[%{TIMESTAMP_ISO8601:log_timestamp}\] +(?<log_level>(?i)(?:debug|notice|warning|error|verbose|dtmf|fax|security)(?-i))\[%{INT:thread_id}\](?:\[%{DATA:call_thread_id}\])? %{DATA:module_name}\: %{GREEDYDATA:log_message}"
                                }
                                add_field => [ "received_timestamp", "%{@timestamp}"]
                                add_field => [ "process_name", "asterisk"]
                        }
                        if ![log_message]
                        {
                mutate
                {
                    add_field => {"log_message" => ""}
                }
            }
            if [log_message] =~ /^Executing/ and [module_name] == "pbx.c"
            {
                grok
                {
                        match =>
                        {
                                "log_message" => "Executing +\[%{DATA:TARGET}@%{DATA:dialplan_context}:%{INT:dialplan_priority}\] +%{DATA:asterisk_app}\(\"%{DATA:protocol}/%{DATA:Ext}-%{DATA:Channel}\",+ \"%{DATA:procedure},%{INT:trunk},%{DATA:dest},,%{DATA:mode}\"\) %{GREEDYDATA:log_message}"
                        }
                }

            }
                }
        }
}
output{
        elasticsearch{
        hosts=>"127.0.0.1:9200"
        index=>"new_asterisk"
}
}

when I check it into kibana index, the index just showing raw logs. Questions: why my conf not parsing logs even the grok I've made successfully tested (by me).

solved

log not get into if condition

Upvotes: 0

Views: 422

Answers (1)

apt-get_install_skill
apt-get_install_skill

Reputation: 2908

It seems like your grok-actions don't get applied at all because the data get indexed raw and no error-tags are thrown. Obviously your documents don't contain a field type with value asterisk_debug which is your condition to execute the grok-actions.

To verify this, you could implement a simple else-path that adds a field or tag indicating that the condition was not met like so:

filter{
  if [type]=="asterisk_debug"{
    # your grok's ...
  }
  else{
    mutate{
      add_tag => [ "no_asterisk_debug_type" ]
    }
  }
}

Upvotes: 1

Related Questions