CODEWITHSUNDEEP
filterlogstash
stefansaye
stefansaye

Reputation: 135

logstash name fields dynamically

i have a dynamical field, the field format looks like 

A-B-C::D_[randomNum]

the field is dynamic because the randonNUM , i want to change the '-' to '_' and remove the [randomNUM] and it's will be looks like as follow, 

  A_B_C::D

Is there any plugin / strategy to solve this problem?

Upvotes: 1

Views: 1787

Answers (1)

Val
Val

Reputation: 217564

You should be able to achieve this with a mutate/gsub filter

filter {
  mutate {
    gsub => [
      # replace random num suffix
      "fieldname", "_\d+", "",
      # replace all dashes with underscores
      "fieldname", "-", "_"
    ]
  }
}

Make sure to replace fieldname with your actual field name.

UPDATE

Given your comments, it turned out it's the field names that are dynamic and not the value. For this reason, you cannot use the above solution but the next one should work, i.e. using the ruby filter:

filter {
  ruby {
    code => "
      newhash = {}
      event.to_hash.each {|key, value| 
        if key =~ /^CISCO/ then
            newkey = key.gsub(/_\d+/, '').gsub('-', '_')
            newhash[newkey] = event[key]
            event.remove(key)
        end
      }
      newhash.each {|key,value|
        event[key] = value
      }
    "
  }
}

After this filter runs, your event will have the field A_B_C::D instead of the original A-B-C::D_num

Upvotes: 1

Related Questions