CODEWITHSUNDEEP
pythonpython-3.xprometheus
PythonNewbie
PythonNewbie

Reputation: 1163

How to run multiple python scripts to prometheus

I have been working on prometheus and Python where I want to be able to have multiple scripts that writes to Promethethus.

Currently I have done 2 scripts: sydsvenskan.py

import time

import requests
from prometheus_client import Counter

REQUEST_COUNT = Counter(
    namespace="scraper",
    name="request_count",
    documentation="Count the total requests",
    labelnames=['http_status']
)


def monitor_feed():
    while True:
        with requests.get("https://sydsvenskan.se") as rep:
            print("Request made!")
            REQUEST_COUNT.labels(http_status=rep.status_code).inc()

        time.sleep(10)


if __name__ == '__main__':
    monitor_feed()

BBC.py

import time

import requests
from prometheus_client import Counter

REQUEST_COUNT = Counter(
    namespace="scraper",
    name="request_count",
    documentation="Count the total requests",
    labelnames=['http_status']
)


def monitor_feed():
    while True:
        with requests.get("https://bbc.com") as rep:
            print("Request made!")
            REQUEST_COUNT.labels(http_status=rep.status_code).inc()

        time.sleep(10)


if __name__ == '__main__':
    monitor_feed()

and then I have another script that just starts the promethethus http_server:

from prometheus_client import start_http_server

if __name__ == '__main__':
    start_http_server(8000)

however the problem is it seems like nothing goes through the promethethus from the sydsvenskan.py and bbc.py and I wonder what am I doing wrong? I do not see any statistics growing when running the sydsvenskan and bbc at the same time

Upvotes: 0

Views: 609

Answers (1)

DazWilkin
DazWilkin

Reputation: 40091

You need to combine the start_http_server function with your monitor_feed functions.

You can either combine everything under a single HTTP server.

Or, as I think you want, you'll need to run 2 HTTP servers, one with each monitor_feed:

import time
import requests

from prometheus_client import Counter
from prometheus_client import start_http_server


REQUEST_COUNT = Counter(
    namespace="scraper",
    name="request_count",
    documentation="Count the total requests",
    labelnames=['http_status']
)


def monitor_feed():
    while True:
        with requests.get("https://bbc.com") as rep:
            print("Request made!")
            REQUEST_COUNT.labels(http_status=rep.status_code).inc()

        time.sleep(10)


if __name__ == '__main__':
    start_http_server(8000)
    monitor_feed()

In the latter case, if you run both servers on the same host machine, you'll need to use 2 different ports (you can't use 8000 for both).

Upvotes: 1

Related Questions