CODEWITHSUNDEEP
jsonnode.jsmonitoringprometheus
Joshua Alger
Joshua Alger

Reputation: 2132

Does Prometheus allow you to scrape JSON information from an endpoint?

I am using Prometheus to instrument a Node.js application for monitoring. I am currently using the following Node.js client for instrumentation:

prom-client

I have everything configured to gather and collect default metrics from my Node.js application and the monitoring is working as expected. I want to know if it is possible for Prometheus to scrape JSON from an endpoint that my application exposes.

For example, the Node.js application has a health check endpoint (/health), that returns simple JSON data (boolean values or 0/1) about the overall health of the application as well as it's dependencies. Can I configure Prometheus and/or the prom-client to scrape the JSON from the health endpoint and then record metrics based on that information?

Upvotes: 9

Views: 42050

Answers (5)

Dmytro Kutetskyi
Dmytro Kutetskyi

Reputation: 799

Now it's possible with the blackbox exporter: https://github.com/prometheus/blackbox_exporter.

Here is step by step guide for prometheus/grafana/blackbox - https://medium.com/the-telegraph-engineering/how-prometheus-and-the-blackbox-exporter-makes-monitoring-microservice-endpoints-easy-and-free-of-a986078912ee.

Upvotes: 1

Joshua Alger
Joshua Alger

Reputation: 2132

I was able to find a solution using the prom-client and building my own custom metric. Will provide an example below for anyone who may be interested in doing the same. Let's say that there is a health check endpoint that returns the following JSON:

{
    "app": {
        "message": "Service is up and running!",
        "success": true
    }
}

I used the package request to make a call to the endpoint, parse the data and create a gauge to reflect a value based on the health check status. Below is an example of the /metrics endpoint in JavaScript:

const express = require('express');
const router = express.Router();
const request = require('request');

// Config for health check endpoint
const healthCheckURL = 'https://SOME_ENDPOINT/health';
const zone = 'DEV';

// Initialize Prometheus
const Prometheus = require('prom-client');
const collectDefaultMetrics = Prometheus.collectDefaultMetrics;
collectDefaultMetrics({
    timeout: 5000
});

router.get('/', (req, res) => {
    res.end(Prometheus.register.metrics());
});

const serviceHealthGauge = new Prometheus.Gauge({
    name: 'service_health',
    help: 'Health of service component',
    labelNames: ['zone']
});

setInterval(() => {
    request({
            url: healthCheckURL,
            method: "GET",
        },
        function(error, response, body) {
            if (!error && response.statusCode == 200) {
                const JSONBody = JSON.parse(body);

                // check service health
                if (JSONBody.app && JSONBody.app.success) {
                    serviceHealthGauge.set({
                        zone: zone
                    }, 1);
                } else {
                    serviceHealthGauge.set({
                        zone: zone
                    }, 0);

                }
            } else {
                serviceHealthGauge.set({
                    zone: zone
                }, 0);
            }
        }   
    );
  }, 10000);

module.exports.metricNames = ['service_health'];

module.exports = router;

Upvotes: 5

ahus1
ahus1

Reputation: 5940

If you want to have the prom-client to collect this information, you could have a look at the heap sizes collector that is part of the library.

Where this collector fetches the heap size you could instead scrape the JSON endpoint or maybe call the functionality behind the JSON endpoint directly to publish some gauges with either 0 or 1.

Upvotes: 2

Heiko Rupp
Heiko Rupp

Reputation: 30984

Not directly, as Prometheus only understands their text format or a GRPC one. See https://prometheus.io/docs/instrumenting/exposition_formats/

Or course it is possible to write a translation "bridge" or exporter that translates JSON structures in that format, like @ConorB described in his answer.

Upvotes: 2

Conor
Conor

Reputation: 3579

I believe you can.

The blogposts I have linked below detail how this is done using the Prometheus Python client to ingest metrics in JSON format into Prometheus.

https://www.robustperception.io/writing-a-jenkins-exporter-in-python/ https://www.robustperception.io/writing-json-exporters-in-python/

Upvotes: 9

Related Questions