CODEWITHSUNDEEP
logstashelk
linnur
linnur

Reputation: 1

How to remove unwanted fields logstash (ELK)

How can I remove unnecessary fields? Type: agent.ephemeral_id agent.id winlog.provider_guid

I tried, but Kibana stops showing logs at all

- drop_fields:
      fields: ["date_created", "ecs.version", "agent.version", "agent.type", "agent.id"]

In logstash I have these configs: filter.conf, input.conf, output.conf

Filter:

filter {
    if "winsrvad" in [tags] {
    if [winlog][event_id] != "5136" and [winlog][event_id] !=ent_id] != "4729" and id] != "4734" {
    drop { }
    }
    }
}

Upvotes: 0

Views: 2529

Answers (3)

Minh Vu
Minh Vu

Reputation: 51

The drop option will ignore the entire log, you can use remove_field option inside the mutate plugin instead.

For example:

mutate {
  remove_field => [
    "date_created",
    "[ecs][version]",
    "[agent][version]",
    "[agent][type]",
    "[agent][id]"
  ]
}

I have written a detailed tutorial for this: How to Remove Field in Logstash.

Upvotes: -2

Ashok
Ashok

Reputation: 1

You can use mutate;like below

  mutate {
    remove_field => [ "date_created", "ecs.version", "agent.version", "agent.type", "agent.id"]
  }

Upvotes: 0

Shiva Janssens
Shiva Janssens

Reputation: 71

I would suggest using prune to blacklist the 'unnecessary fields' when the condition is met.

See documentation: https://www.elastic.co/guide/en/logstash/current/plugins-filters-prune.html

Upvotes: 1

Related Questions