CODEWITHSUNDEEP
logstashelastic-stack
ab11
ab11

Reputation: 20100

Logstash doesnt read from configured input file

I am trying to configure my Logstash to read from a specified log file. When I configure it to read from stdin it works as expected, my input results in a message from Logstash and displays in my Kibana UI.

$ cat /tmp/logstash-stdin.conf 
input {
  stdin {}
}
output {
  elasticsearch { hosts => ["localhost:9200"] }
  stdout { codec => rubydebug }

}

$./logstash -f /tmp/logstash-stdin.conf
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path //usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
The stdin plugin is now waiting for input:
hellloooo
{
      "@version" => "1",
          "host" => "myhost.com",
    "@timestamp" => 2017-11-17T16:05:41.595Z,
       "message" => "hellloooo"
}

However, when I run Logstash with a file input I get no indication that the file is loaded into Logstash, and it does not show in Kibana. 

$ cat /tmp/logstash-simple.conf 
input {
  file {
    path => "/tmp/test_log.txt"
    type => "syslog"
  }
}
output {
  elasticsearch { hosts => ["localhost:9200"] }
  stdout { codec => rubydebug }
}


$ ./logstash -f /tmp/logstash-simple.conf 
WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path //usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console

Any suggestions of how I can troubleshoot why my Logstash is not ingesting the configured file?

Upvotes: 1

Views: 1442

Answers (1)

mustaccio
mustaccio

Reputation: 19021

By default the file input plugin starts reading at the end of the file, so only lines added after Logstash starts will be processed. To read all existing lines upon startup add the option "start_position" => "beginning" to the configuration, as explained in documentation.

Upvotes: 1

Related Questions