CODEWITHSUNDEEP
rubylogstash
mikeb
mikeb

Reputation: 11267

Logstash Error - how to trace

I'm getting the following in my Logstash log, and can't figure out where it's coming from:

{:timestamp=>"2016-05-19T12:14:00.883000-0400", :message=>"Ruby exception occurred: undefined method `[]' for nil:NilClass", :level=>:error}

I have some other errors, too and I can't tell which record is causing it. I suspect my error here is from the date manipulation, but the question is more general.

How can I make logstash print the entire event that causes this error, or any similar one?

My filter an output look like this, the input is from a JDBC source:

filter {
  mutate
  {
    convert => [ "dt", "string" ]
  }
  date {
    match => [ "dt", "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"]
  }
  ruby {
    code => "event['idx'] = event['dt'][0..9]"
  }
  ruby {
    code => "event['id'] = event['@timestamp'].time().to_f.to_s + '' + event['sessionid']"
  }

}

output {
  stdout {
    codec => rubydebug
  }
  elasticsearch {
    hosts => ["localhost" ]
    index => "prefix-%{idx}"
    document_id => "%{id}"
  }
}

Upvotes: 3

Views: 3242

Answers (1)

alpert
alpert

Reputation: 4655

It seems that this line is in filter method of ruby filter:

def filter(event)
  begin
    @codeblock.call(event)
    filter_matched(event)
  rescue Exception => e
    @logger.error("Ruby exception occurred: #{e}")
    event.tag("_rubyexception")
  end
end

I think you can add a error message to print the event. So you will know which event cause the error:

def filter(event)
  begin
    @codeblock.call(event)
    filter_matched(event)
  rescue Exception => e
    @logger.error("Ruby exception occurred: #{e}")
    @logger.error("Event causing error: #{event}") # <= Log event
    event.tag("_rubyexception")
  end
end

It is in file: vendor/bundle/jruby/1.9/gems/logstash-filter-ruby-2.0.5/lib/logstash/filters/ruby.rb for Logstash 2.3.2

Upvotes: 1

Related Questions