CODEWITHSUNDEEP
loggingelasticsearchbackfillelastic-stack
Celi Manu
Celi Manu

Reputation: 471

ELK stack - How do i backfill all old logs into elasticsearch?

Ive got ELK + filebeat set up with logs coming in, but want to backfill all the old logs into logstash/ES. How can I do this? People have mentioned deleting sincedb file and/or adding this to my logstash input.conf:

  file {
      path => "/var/log/xx/xx.log"
      start_position => "beginning"
      sincedb_path => "/dev/null"
     }

But I added it to my input conf and restarted logstash and still don't see old logs in Kibana. I also can't find the sincedb_* file that everyone refers to. My ELK node is a RHEL server.

Thanks

Upvotes: 2

Views: 1336

Answers (1)

Val
Val

Reputation: 217274

If those files are old, it's a good idea to also add ignore_older => 0 to your config.

  file {
      path => "/var/log/ptsfd-mms/ptsfd-mms.log"
      start_position => "beginning"
      sincedb_path => "/dev/null"
      ignore_older => 0
  }

Upvotes: 1

Related Questions