CODEWITHSUNDEEP
logstash
Milor123
Milor123

Reputation: 555

How use regex in logstash input file

I am trying add multiple logs files to my logstash for load all index data in kibana (according to my regular expression https://regex101.com/r/njG6Qq/2).

This is my /etc/logstash/conf.d/apache-01.conf It appears not work because the index be dont shows in in kibana

input {
  file {
    path => "/var/lib/jenkins/workspace/GetLogs/(.+\.)?themaindomain\.com-ssl_log-.+[0-9]{4}$"
    type => "apache_access"
    sincedb_path => ["/var/lib/logstash/"]
    start_position => "beginning"
  }
}

Example of my logs files in contain in /var/lib/jenkins/workspace/GetLogs/ folder like my regex https://regex101.com/r/njG6Qq/2

somesubdomain.themaindomain.com-Nov-2018
somesubdomain.themaindomain.com-Oct-2018
somesubdomain.themaindomain.com-Sep-2018
somesubdomain.themaindomain.com-Sep-2018.gz.1
somesubdomain.themaindomain.com-ssl_log-Jan-2018
somesubdomain.themaindomain.com-ssl_log-Jan-2018.gz.1
somesubdomain.themaindomain.com-ssl_log-Nov-2018
somesubdomain.themaindomain.com-ssl_log-Oct-2018
somesubdomain.themaindomain.com-ssl_log-Sep-2018
somesubdomain.themaindomain.com-ssl_log-Sep-2018.gz.1
ftp.themaindomain.com-ftp_log-Mar-2018
ftp.themaindomain.com-ftp_log-Mar-2018.gz.1
ftp.themaindomain.com-ftp_log-Oct-2018
ftp.themaindomain.com-ftp_log-Sep-2018
merged.txt
somesubdomain.themaindomain.com-Oct-2018
somesubdomain.themaindomain.com-Sep-2018
somesubdomain.themaindomain.com-Sep-2018.gz.1
somesubdomain.themaindomain.com-ssl_log-Oct-2018
somesubdomain.themaindomain.com-ssl_log-Sep-2018
somesubdomain.themaindomain.com-ssl_log-Sep-2018.gz.1
OTHERsubdomain.themaindomain.com-Sep-2018
OTHERsubdomain.themaindomain.com-Sep-2018.gz.1
OTHERsubdomain.themaindomain.com-ssl_log-Sep-2018
OTHERsubdomain.themaindomain.com-ssl_log-Sep-2018.gz.1
somesubdomain.themaindomain.com-Nov-2018
somesubdomain.themaindomain.com-Oct-2018
somesubdomain.themaindomain.com-Sep-2018
somesubdomain.themaindomain.com-Sep-2018.gz.1
somesubdomain.themaindomain.com-ssl_log-Nov-2018
somesubdomain.themaindomain.com-ssl_log-Oct-2018
somesubdomain.themaindomain.com-ssl_log-Sep-2018
somesubdomain.themaindomain.com-ssl_log-Sep-2018.gz.1
OTHERsubdomain.themaindomain.com-Jun-2018
OTHERsubdomain.themaindomain.com-Jun-2018.gz.1
OTHERsubdomain.themaindomain.com-May-2018
OTHERsubdomain.themaindomain.com-May-2018.gz.1
OTHERsubdomain.themaindomain.com-ssl_log-Jun-2018
OTHERsubdomain.themaindomain.com-ssl_log-Jun-2018.gz.1
OTHERsubdomain.themaindomain.com-ssl_log-May-2018
OTHERsubdomain.themaindomain.com-ssl_log-May-2018.gz.1
somesubdomain.themaindomain.com-Nov-2018
somesubdomain.themaindomain.com-Oct-2018
somesubdomain.themaindomain.com-Sep-2018

how should I add the regex sentence to the logstash configure file? Can someone explain me? Thank you very much

Upvotes: 0

Views: 1810

Answers (1)

ben5556
ben5556

Reputation: 3018

You can only use filename patterns in path. In your case use /var/lib/jenkins/workspace/GetLogs/*.themaindomain.com-ssl_log-???-????

Documentation shows only filename patterns can be used - https://www.elastic.co/guide/en/logstash/current/plugins-inputs-file.html#plugins-inputs-file-path

For more information on file patterns see the below link

https://www.ibm.com/support/knowledgecenter/en/SSMKHH_10.0.0/com.ibm.etools.mft.doc/ac55200_.htm

Upvotes: 1

Related Questions