CODEWITHSUNDEEP
prometheusmonitoringendpoint
cr001
cr001

Reputation: 663

Prometheus monitoring endpoint access number : is this possible?

I have an HTTP API server with endpoints like https://example.com/api/login and https://example.com/api/logout . etc

I have been looking for a way to monitor each API's access number every minutes and record them as a graph.

The easy way I was doing was basically adding some server code to record each access to a log file during access and then scraping through the log file.

However I think maybe prometheus can do this as well and this would be a much more proper way and more maintainable in the future.

I am completely new to prometheus and I did some research and found out maybe the blackbox exporter can do what I want. However the documentation seems to indicate that it will only "probe" the endpoint, which based on my understanding is to test the stability/availability parameters of the endpoint, rather than recording the number of access to each endpoint.

Will "recording access number for specified endpoints" be possible with prometheus? Or other open-source monitoring tools?

Upvotes: 4

Views: 1722

Answers (3)

Simon Gurcke
Simon Gurcke

Reputation: 491

A simpler alternative to using Prometheus for monitoring requests to your API endpoints is Apitally, if your API is built with one of the supported web frameworks.

Apitally's open-source client libraries implement middleware for various Python & Node.js frameworks, which captures request & response metadata and asynchronously ships it to its servers in 1 minute intervals.

The Apitally dashboard then provides insights into API traffic, errors, response times and payload sizes, for the whole API and individual endpoints.

To set this up, you would just have to install a small dependency and add a couple of lines of code to your project.

Disclaimer: I'm the author of Apitally and have built it specifically for this kind of use case.

Upvotes: 0

anemyte
anemyte

Reputation: 20176

Prometheus gathers data from HTTP endpoints, meaning that it makes a HTTP GET request and the response should contain data in OpenMetrics format. Prometheus cannot read logs to create metrics.

However, there are lots of exporters and some of them can create metrics from logs. You can find an incomplete list here: https://prometheus.io/docs/instrumenting/exporters/#logging . The setup will be as following:

  • you install an exporter next to your logs and configure it to read them and make metrics;
  • you install Prometheus somewhere and configure it to scrape the exporters;
  • then you can use Prometheus UI or Grafana to view graps etc.

An alternative way will be to integrate Prometheus library into that API of yours. Then you can count requests internally and create an API endpoint for Prometheus to visit. In my opinion this is better than metrics via logs - less software, less things that can break.

At last, you can consider using Loki (https://grafana.com/oss/loki/) or the Elastic Stack (https://www.elastic.co/elastic-stack/). These are tools to gather logs but you can also use them to create dashboards, build graphs, etc.

Upvotes: 2

crock
crock

Reputation: 461

If you are self-monitoring your Prometheus instance(s), there is a metric called prometheus_http_requests_total which exposes the total HTTP requests for each endpoint.

If you don't know what I mean by self-monitoring your Prometheus, just add a scrape job like this:

  - job_name: 'prometheus'
    static_configs:
    - targets:
      - localhost:9090 # your Prometheus port

Upvotes: 0

Related Questions