CODEWITHSUNDEEP
graphitegrafanastatsd
Sean Lindo
Sean Lindo

Reputation: 1437

How do you sum a statsd counter over a large time range with correct values?

Background

A basic use case for statsd & grafana is learning how many times a function has been called over a time range-- whether that is "last 6h", "since beginning of today", "since beginning of time", etc.

What I'm struggling to find is the correct function to achieve this. I'm using a hosted solution; however, I can confirm that data is being flushed from StatsD to Graphite in 10s intervals.

Current Setup

StatsD Flush: 10s

Graph Function: hitcount(counters.login.employer.count, "10seconds")

Time Range: 24h

Problem

When using hitcount(counters.login.employer.count, "10seconds"), the data returned is incorrect. In fact, I can do 24h, 23h, 22h, and note the values are actually increasing.

I've performed all testing here in a controlled environment, only my machine is sending metrics to StatsD. This is not yet in production code.

Any idea what could be going on here?

Upvotes: 1

Views: 3096

Answers (1)

AussieDan
AussieDan

Reputation: 2176

The way counters work is that on each interval the value of the counter is sent to graphite and reset in statsd, so what you're looking for is the sum of the series.

You can do that using consolidateBy('sum') combined with maxDataPoints=1.

Be aware that if your series is being aggregated in graphite you'll need to make sure that the aggregation is by sum, otherwise when values get rolled up from the individual values reported by statsd into aggregated buckets they'll be averaged, and your sum won't work across longer intervals. You can read more about configuring aggregation in Graphite here.

Upvotes: 1

Related Questions