CODEWITHSUNDEEP
logstash
Gabor Sandor
Gabor Sandor

Reputation: 53

Logstash get only message content

I would like to read a file line-by-line and put only the rows into a queue. The process works but in the queue there are a lot of other information such as version and so one.

Do you have any idea?

Here is my config file:

input {
   file {
     codec => plain
     path => "/apps/*.txt"
  }
}
filter{

}
output {
    stomp {
        host => "localhost"
        port => "61613"
        destination => "stomp-test"
        debug => "true"
        headers => {
                "amq-msg-type" => "text"
                "my_custom_header" => "kutya"
        }
    }
}

The example.txt contains: This is my test row!

The MQ body contains: {"message":"This is my test row!","@version":"1","@timestamp":"2017-01-21T13:42:21.084Z","path":"/apps/1.txt","host":"ol7"}

My expected result should be: This is my test row!

Thanks in advance.

Upvotes: 1

Views: 3216

Answers (2)

Barry Hatfield
Barry Hatfield

Reputation: 41

I came across this question looking for a similar solution.

Your question can be solved using codec.

Add this line to your output:

      codec => line { format => "%{message}"}

It will look like this:

output {
    stomp {
        host => "localhost"
        port => "61613"
        destination => "stomp-test"
        debug => "true"
        codec => line { format => "%{message}"}
        headers => {
                "amq-msg-type" => "text"
                "my_custom_header" => "kutya"
        }
    }
}

Upvotes: 3

Val
Val

Reputation: 217354

The stomp output, as it currently stands, will only write JSON to the destination queue as can be seen in the source code of that plugin:

t.send(event.sprintf(@destination), event.to_json, headers)
                                          ^
                                          |
                                     writes JSON

The best you can do is to remove all other fields using a mutate/remove_field filter, like this:

filter {
  mutate {
    remove_field => ["@timestamp", "@version", "path", "host"]
  }
}

Then you'll get only this in your destination queue:

{"message":"This is my test row!"}

Upvotes: 1

Related Questions