CODEWITHSUNDEEP
logginggraphite
seanmcl
seanmcl

Reputation: 9946

show graphite invalid lines

I see lots of lines like this in my graphite logs:

01/10/2014 21:07:12 :: [listener] invalid line received from client HOST:PORT, ignoring

It would greatly help if I could see the invalid line. Some documentation and tutorials suggest graphite would print the offending line directly after the invalid warning, but for me it doesn't. How can I enable this property?

Thanks.

Upvotes: 3

Views: 2258

Answers (1)

John T Dyer
John T Dyer

Reputation: 156

So my attempt to troubleshoot this was a total hack but it worked for me.

Steps

  • Edit protocol.py (/opt/graphite/lib/carbon/protocols.py on line 75 and add an additional log line
Before: 
 class MetricLineReceiver(MetricReceiver, LineOnlyReceiver):
    delimiter = '\n'

  def lineReceived(self, line):
    try:
      metric, value, timestamp = line.strip().split()
      datapoint = (float(timestamp), float(value))
    except:
      log.listener('invalid line received from client %s, ignoring' % self.peerName )
      return

    self.metricReceived(metric, datapoint)
After: 
 class MetricLineReceiver(MetricReceiver, LineOnlyReceiver):
    delimiter = '\n'

  def lineReceived(self, line):
    try:
      metric, value, timestamp = line.strip().split()
      datapoint = (float(timestamp), float(value))
    except:
      log.listener('invalid line received from client %s, ignoring' % self.peerName )
      log.listener('invalid line - [ %s ]' % line)
      return

    self.metricReceived(metric, datapoint)

  • restart daemon in debug mode

    /usr/bin/python /opt/graphite/bin/carbon-cache.py --pid /opt/graphite/storage/carbon-cache-a.pid --debug start

  • find problem metric and fix
  • revert change to protocol.py
  • restart carbon-cache as daemon

By doing this I was able to see exactly which metric was causing me grief and address it.

Hope that helps!

Upvotes: 7

Related Questions