CODEWITHSUNDEEP
logstashlogstash-configuration
BJH
BJH

Reputation: 45

Using regex in logstash to filter files by name

I am pulling in a series of files and using logstash to filter out only the ones i need based on a regex before outputting the matched files to a named directory, but the regex is not working and nothing is being filtered out.

output {
    if [filename] =~ /.*[abc|def].*/
 {
        file {
        path => "/my/directory/%{filename}-%{+YYYY-MM-dd}.log"
        codec => line { format => "%{message}"}
        }
    }
}

Incoming files are of the format:

P01_abc_stdout.log-2020-11-10-11.log
P01_rtf_stdout.log-2020-11-10-11.log
P01_ccc_stdout.log-2020-11-10-11.log
P01_def_stdout.log-2020-11-10-11.log
P01_ces_stdout.log-2020-11-10-11.log

but all of these files have come through, even though I'd expect it only to match

P01_abc_stdout.log-2020-11-10-11.log
P01_def_stdout.log-2020-11-10-11.log

Upvotes: 0

Views: 1031

Answers (1)

Badger
Badger

Reputation: 4072

if [filename] =~ /.*[abc|def].*/

Square brackets define a character group, so that will match any file that contains one of the characters a through f, or pipe. All of your filenames contain stdout, which includes a d, so they all match. The leading and trailing .* are not needed, since the pattern is not anchored. Try

if [filename] =~ /(abc|def)/

Upvotes: 1

Related Questions