CODEWITHSUNDEEP
elasticsearchjmeterlogstash
Any1
Any1

Reputation: 182

Logstash not writing to elasticsearch (_discover_file_glob)

Hey i used followed this turorial for setting up jmeter & logstash with kibana for elasticsearch: http://ecmarchitect.com/archives/2014/09/09/3932

The first time everything worked. A new jmeter-results index was created and filled by logstash with my jmeter data. Today I tried the same with new jmeter data, but nothing happened. No Error occurred, but in the logstash logs i could see that _discover_file_glob was logged over and over again.Here is the important part of my log: 

 Registering file input {:path=>["/etc/apache-jmeter-2.12/bin/log2.jtl"], :level=>:info, :file=>"logstash/inputs/file.rb", :line=>"74"}
No sincedb_path set, generating one based on the file path {:sincedb_path=>"/root/.sincedb_66c8ea3a6e5fbda3879299a795b893d5", :path=>["/etc/apache-jmeter-2.12/bin/log2.jtl"], :level=>:info, :file=>"logstash/inputs/file.rb", :line=>"115"}
Pipeline started {:level=>:info, :file=>"logstash/pipeline.rb", :line=>"78"}
_sincedb_open: reading from /root/.sincedb_66c8ea3a6e5fbda3879299a795b893d5 {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"199"}
_sincedb_open: setting [33297239, 0, 2306] to 44106 {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"203"}
_discover_file_glob: /etc/apache-jmeter-2.12/bin/log2.jtl: glob is: ["/etc/apache-jmeter-2.12/bin/log2.jtl"] {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"}
_discover_file: /etc/apache-jmeter-2.12/bin/log2.jtl: new: /etc/apache-jmeter-2.12/bin/log2.jtl (exclude is []) {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"126"}
_open_file: /etc/apache-jmeter-2.12/bin/log2.jtl: opening {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"98"}
/etc/apache-jmeter-2.12/bin/log2.jtl: sincedb last value 44106, cur size 44106 {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"122"}
/etc/apache-jmeter-2.12/bin/log2.jtl: sincedb: seeking to 44106 {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"124"}
writing sincedb (delta since last write = 1421612560) {:level=>:debug, :file=>"filewatch/tail.rb", :line=>"177"}
/etc/apache-jmeter-2.12/bin/log2.jtl: file grew, old size 0, new size 44106 {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"81"}
Automatic template management enabled {:manage_template=>"true", :level=>:info, :file=>"logstash/outputs/elasticsearch_http.rb", :line=>"104"}
Template Search URL: {:template_search_url=>"http://localhost:9200/_template/*", :level=>:debug, :file=>"logstash/outputs/elasticsearch_http.rb", :line=>"112"}
_discover_file_glob: /etc/apache-jmeter-2.12/bin/log2.jtl: glob is: ["/etc/apache-jmeter-2.12/bin/log2.jtl"] {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"}
_discover_file_glob: /etc/apache-jmeter-2.12/bin/log2.jtl: glob is: ["/etc/apache-jmeter-2.12/bin/log2.jtl"] {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"}
_discover_file_glob: /etc/apache-jmeter-2.12/bin/log2.jtl: glob is: ["/etc/apache-jmeter-2.12/bin/log2.jtl"] {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"}
_discover_file_glob: /etc/apache-jmeter-2.12/bin/log2.jtl: glob is: ["/etc/apache-jmeter-2.12/bin/log2.jtl"] {:level=>:debug, :file=>"filewatch/watch.rb", :line=>"117"}

I read in the internet that the solution is to delete the .sincedb_ files , but still nothing happens.

Maybe anyone can help me?

Upvotes: 1

Views: 707

Answers (1)

VNaik
VNaik

Reputation: 48

Set start_position to beginning in input section of logStash to start processing same CSV file again:

input {

  file {
    path => [ "/CSV_File.csv"]
    type => "JMeterlog"
    start_position => "beginning"
    sincedb_path => "/dev/null"

 }
}
filter {
if ([message] =~ "responseCode") {
        drop { }
} 
else { 
        csv { columns => ["timeStamp", "elapsed", "label", "responseCode", "responseMessage", "threadName", "dataType", "success", "bytes", "grpThreads", "allThreads", "URL", "Latency", "SampleCount", "ErrorCount", "IdleTime"]}
     }
}
output {
  stdout { codec => rubydebug }

  elasticsearch {
    hosts => ["localhost:9200"]
    index => "logstash-jmeter-results-%{+YYYY.MM.dd}"
    template => "jmeter-results-mapping.json"
    template_name => "logstash-jmeter-results"
    template_overwrite => false
    }
}

Upvotes: 1

Related Questions