CODEWITHSUNDEEP
logstashelasticsearch
Phil B
Phil B

Reputation: 6057

Logstash doesn't write / process data and seems to fail silently

Logstash seems to hang when processing a local file. The logstash process is still alive and everything looks fine, but no data get written to the output (elasticsearch). The index gets written, though.

Upvotes: 0

Views: 264

Answers (1)

Phil B
Phil B

Reputation: 6057

Logstash seems to "hang" and not process any of the input data for the following reason: Logstash keeps track of what has previously been processed, so when you run it again on the same input data (as will be the case during testing), Logstash will think it has already seen and processed this data the previous time and will not read it again. To bypass this during testing, specify explicitly the location of the sincedb file where Logstash should keep track of what it has read or not and manually delete this sincedb file before each test run.

Here is an example:

input {
    file {
        path => "~/logstash/data/input_file"
        start_position => "beginning"
        sincedb_path => "~/logstash/data/sincedb.db"
    }
}

or maybe even better (added based on comment below):

input {
    file {
        path => "~/logstash/data/input_file"
        start_position => "beginning"
        sincedb_path => "/dev/null"
    }
}

Upvotes: 1

Related Questions