CODEWITHSUNDEEP
logstash
Forece85
Forece85

Reputation: 518

KV filter not including space values

The logstash kv filter is not including values with spaces. Please check below the input and output message generated by Logstash. The desired output should include spaces.

Input message:

key1=first value key2=second value key3=value3

Desired output:

{
    "key1" => "first value",
    "key2" => "second value",
    "key3" => "value3",
    "message" => "key1=first value key2=second value key3=value3"
}

Obtained output:

{
    "key1" => "first",
    "key2" => "second",
    "key3" => "value3",
    "message" => "key1=first value key2=second value key3=value3"
}

I need spaces in values to be included in fields. How to do this in Logstash?

Upvotes: 1

Views: 3200

Answers (1)

baudsp
baudsp

Reputation: 4110

The kv filter separates the key-value pairs (by default) with spaces. So when parsing key1=first value key3=value3, it will pick key1=first, value and key3=value3. value is discarded since it does not have the key-value separator (=), then the two pairs are treated, giving "key1" => "first" & "key3" => "value3".

This can't be fixed with the kv filter; you'll have to modify the input before sending it to kv, to something like this: key1=first value, key2=second value, key3=value3 => with this you'll be able to use , as field_split in the kv configuration.

You can use mutate before the kv filter to add , between the key-value pairs like this:

mutate {
    gsub => ["message", "(\S+=)", ", \1"]
}

This will replace all non-whitespace characters before the = sign with itself preceded by , .

Then use:

kv {
    field_split => ","
}

Upvotes: 5

Related Questions