CODEWITHSUNDEEP
logstash
laymo
laymo

Reputation: 17

What option can be chosen in logastash

I'm using logstash to parse logs from files, my question is witch option should i choose to collect all my logs without duplicating the data

 start_position => 'beginning'

or

 start_position => 'end'

and what is the deference between the two options. Thank's

Upvotes: 1

Views: 65

Answers (1)

Alcanzar
Alcanzar

Reputation: 17165

To quote from the docs:

Choose where Logstash starts initially reading files: at the beginning or at the end. The default behavior treats files like live streams and thus starts at the end. If you have old data you want to import, set this to ‘beginning’

This option only modifies “first contact” situations where a file is new and not seen before. If a file has already been seen before, this option has no effect.

So if you always want to import the whole file, use beginning. If what you are dealing with is some sort of streaming situation where only the latest stuff matters, use end. It's important that you use beginning if you have a log file that changes names from day to day. Otherwise when logstash sees the next days file for the first time, it'll skip over the beginning of it.

I personally can't think of any situation where you wouldn't want to use beginning, but then maybe I'm dense.

Upvotes: 1

Related Questions