CODEWITHSUNDEEP
syslog-ng
user11890100
user11890100

Reputation: 11

Rewriting log data

I am sending syslog data to my LogZilla server and am unable to rewrite the data using the 

Event message: 

          {"event_type":"Threat_Event","ipv4":"172.31.100.13","hostname":"server1.something.net","source_uuid":"df4df304c3-93f2a-41f89-8dfefd-7f54bdsf5e429f","occured":"06-Aug-2019 02:38:44","severity":"Warning","threat_type":"test file","threat_name":"Eicar","scanner_id":"Real-time file system protection","engine_version":"1498036 (20190805)","object_type":"file","object_uri":"file:///home/admin/g4.txt","action_taken":"cleaned by deleting","threat_handled":true,"need_restart":false,"username":"root","processname":"/usr/bin/vi","circumstances":"Event occurred on a newly created file.","firstseen":"06-Aug-2019 02:38:44","hash":"CF8BD9DFDDFF007F75ADF4C2BE48005CEA317C62"}

Code for automatic key value detection to rewrite message above:

{
  "rewrite_rules": [
    {
      "match": {
        "field": "program",
        "value": "ESServer"
      },
      "update": {
          "message": "${event_type}, ${ipv4}"
      },

      "kv": {"separator": ":", "delimiter": ","
      }

    }
  ] 
  }

'''

I am expecting the message to be parsed so that I can setup dashboards based on various fields from the message.

Upvotes: 1

Views: 1973

Answers (1)

Tom Damon
Tom Damon

Reputation: 748

LogZilla doesn't parse kv pairs within quotes, so first you'll need to strip those out. Here's a syslog-ng rule that will do that:

filter f_program {program("ESServer")};

rewrite r_quotes { subst("\"", "", value("MESSAGE") flags("global") condition( filter(f_program))); };

log {
    source(s_logzilla);
    rewrite (r_quotes);
    #filter(f_fwdrops);
    destination(d_logzilla_network);
    # Uncomment line below for debug/testing of incoming events
    #destination(df_debug);
    #destination(d_unix_stream);
    flags(flow-control,final);
};

You should create a 'rules' directory to store any custom configurations in. Save the above in that directory as syslog.conf (or any name you prefer). The copy it to the conainer and restart syslog-ng:

docker cp syslog.conf lz_syslog:/etc/logzilla/syslog-ng
docker restart lz_syslog

Now those events should have the quotes removed when they come in. Next, create a LogZilla parser rule with the following:

first_match_only: true
rewrite_rules:
- comment:
  - 'Name: ESET Security Manager KV'
  - 'Sample: "event_type":"Threat_Event","ipv4":"172.31.100.13","hostname":"server1.something.net","source_uuid":"df4df304c3-93f2a-41f89-8dfefd-7f54bdsf5e429f","occured":"06-Aug-2019 02:38:44","severity":"Warning","threat_type":"test file","threat_name":"Eicar","scanner_id":"Real-time file system protection","engine_version":"1498036 (20190805)","object_type":"file","object_uri":"file:///home/admin/g4.txt","action_taken":"cleaned by deleting","threat_handled":true,"need_restart":false,"username":"root","processname":"/usr/bin/vi","circumstances":"Event occurred on a newly created file.","firstseen":"06-Aug-2019 02:38:44","hash":"CF8BD9DFDDFF007F75ADF4C2BE48005CEA317C62"'
  - 'Description: ESET K/V Detection and User Tag creation'
  match:
    field: program
    op: =~
    value: 'lzadmin'
  kv:
    delimiter: ""
    separator: ":"
    pair_separator: ","
  tag:
    ut_event_type: ${event_type}
    ut_ipv4: ${ipv4}
    ut_hostname: ${hostname}

Then add the rule:

logzilla rules add kv.json

Upvotes: 1

Related Questions