CODEWITHSUNDEEP
javaapache-stormmetricsgraphite
snerd
snerd

Reputation: 1297

Graphite/Carbon how to get per-second metrics

I've dockerized graphite and am working with this library to get metrics from an Apache Storm topology. I'm getting metrics data, but no matter what I do I can only get data per minute where I really need the points to be per second.

enter image description here

As per this SO post I've set the retention policy to grab data every second. I've also set 

conf.put("topology.builtin.metrics.bucket.size.secs", 1);

and 

void initMetrics(TopologyContext context) {
    messageCountMetric = new CountMetric();
    context.registerMetric("digest_count", messageCountMetric, 1);
}

in the class that's setting up the topology and the bolt itself, respectively. To my understanding this should cause metrics to be reported every second. What am I missing here? How can I get metrics to be reported every second?

t/y in advance and happy holidays all!

update 1

here is my storage-schemas.conf file: 

root@cdd13a16103a:/etc/carbon# cat storage-schemas.conf 
# Schema definitions for Whisper files. Entries are scanned in order,
# and first match wins. This file is scanned for changes every 60 seconds.
#
#  [name]
#  pattern = regex
#  retentions = timePerPoint:timeToStore, timePerPoint:timeToStore, ...

# Carbon's internal metrics. This entry should match what is specified in
# CARBON_METRIC_PREFIX and CARBON_METRIC_INTERVAL settings
[carbon]
pattern = ^carbon\.
retentions = 1s:6h,1min:7d,10min:5y


[default_1min_for_1day]
pattern = .*
retentions = 1s:6h,1min:7d,10min:5y


[test]
pattern = ^test.
retentions = 1s:6h,1min:7d,10min:5y

[storm]
pattern = ^storm.
retentions = 1s:6h,1min:7d,10min:5y

Here is my config setup: 

Config conf = new Config();
conf.setDebug(false);
conf.put("topology.builtin.metrics.bucket.size.secs", 1);
conf.registerMetricsConsumer(GraphiteMetricsConsumer.class, 4);
conf.put("metrics.reporter.name", "com.verisign.storm.metrics.reporters.graphite.GraphiteReporter");
conf.put("metrics.graphite.host", "127.0.0.1");
conf.put("metrics.graphite.port", "2003");
conf.put("metrics.graphite.prefix", "storm.test");

Upvotes: 1

Views: 3232

Answers (1)

kwarunek
kwarunek

Reputation: 12577

In order to apply changes in storage-schemas.conf you have to:

  • restart carbons
  • delete old *.wsp or use whisper-resize.py to apply scheme
  • restart carbon-cache
  • make sure that DEFAULT_CACHE_DURATION in webapp's local_settings.py is set to 1
  • make sure nginx/apache2/uwsgi cache is set up correctly as well, if any

There is more whisper-* tools shipped with graphite. The next you may be interested is whisper-info.py

    bash$ whisper-info.py /graphite/whisper/prod/some/metric.wsp 
    maxRetention: 1296000
    xFilesFactor: 0.5
    aggregationMethod: average
    fileSize: 142600

    Archive 0
    retention: 691200
>>  secondsPerPoint: 1
    points: 11520
    size: 138240
    offset: 40

    Archive 1
    retention: 1296000
    secondsPerPoint: 3600
    points: 360
    size: 4320
    offset: 138280

Upvotes: 4

Related Questions