CODEWITHSUNDEEP
syslogfluentdstackdriver
Eddie Greathouse
Eddie Greathouse

Reputation: 185

Fluentd Invalid Time Format with Syslog

I'm attempting to upload "syslogs" created by a java developer to Google's Stackdriver using Bindplane. Bindplane is built off of fluentd.

I'm using a source type of tail. Bindplane is able to read the log file and push it to Stackdriver, but it always appends the following error in the body of the JSON message:

 error: "invalid time format: value = 2019-10-21 16:52:29.588 EDT, error_class = ArgumentError, error = invalid strptime format - `%Y-%m-%d %H:%M:%S.%L%z'"

The format that the logs appear in are slightly different than most syslog formats:

2019-10-21 13:15:02.439 EDT main TRACE Processor.init()

This is what I have in my fluentd config:

<source>
  @type tail
  path C:/Test/Connect_Test/Log/**TestDL**.txt
  pos_file C:/BlueMedora/BindplaneLogAgent/config/449c6ffb-1a29-4172-9058-e19de236d712.tail.log.pos
  tag 449c6ffb-1a29-4172-9058-e19de236d712.1st_auto_policy
  format syslog
  time_format %Y-%m-%d %H:%M:%S.%L%z
</source>

Is there a way I can avoid this error without changing the logs? Since the class/methods belong to another developer, I'm not able to change how the logs are written.

Thanks for any help!

Upvotes: 4

Views: 7614

Answers (1)

AlphaPapa
AlphaPapa

Reputation: 231

Eddie -

I work for Blue Medora, BindPlane is our product. This should fix your issue, if not please let us know and we can help get it configured properly.

Try time_format %Y-%m-%d %H:%M:%S.%L %Z

  1. The lowercase z represents "Time zone as an hour offset from UTC" (eg.+0400)
  2. Capital Z represents "Time Zone Name" which it looks like what you have in your log files.
  3. It also looks like there is a space between the milliseconds and the timezone. So the space should be added as well.

Here's a link to the documentation on the strptime() options that shows difference between %z and %Z

Again, if this doesn't work for you please let us know.

Upvotes: 1

Related Questions