Getting django-eventstream and Pushpin working

Question

I'm trying to get django-eventstream and Pushpin working within a kubernetes cluster. I can successfully subscribe to the stream, but I never receive data sent via send_event. And I do not receive any errors messages to tell me where things are going wrong.

The stream I'm trying to get working is a logging stream to my app, so it is routinely getting updates.

My setup: Using django-eventstream, I have the following in my Django container:

settings.py -

MIDDLEWARE = [
    ...
    'django_grip.GripMiddleware'
]
INSTALLED_APPS = [
    'django_eventstream',
    ...
]
GRIP_URL = 'http://pushpin:5561'

urls.py -

path('event/logmessage/', include(django_eventstream.urls), {'channels': ['logmessage']}),

logutility.py -

import logging

from datetime import datetime
from django_eventstream import send_event
from django.core.serializers import serialize

def logMessage(level, source, message):
    from .models import LogMessage
    logger = logging.getLogger(__name__)
    if level in ['DEBUG', 'INFO', 'WARNING', 'ERROR']:
        level = logging.getLevelName(level)
    logger.debug('Log message received: {0} {1} {2}'.format(logging.getLevelName(level), source, message))
    logger.log(level, (source + ' - ' + message))

    lm = LogMessage.objects.create(msg_time=datetime.now(), level=logging.getLevelName(level), source=source, msg_text=message)
    lmjson = serialize('json', [lm])
    send_event('logmessage', 'message', lmjson)

My web app talks to a frontend container that runs httpd (I know, we're about to switch over to nginx) httpd.conf -

ProxyPass /event/ http://pushpin:7999/event/
ProxyPassReverse /event/ http://pushpin:7999/event/

And my React client does the following:

    componentDidMount() {
        this.getData(this.state.toggleDebug)
        if ('EventSource' in window) {
            let source = new EventSource("/event/logmessage");
            source.onopen = (e) => console.log("SSE connection made", e);
            source.onmessage = (e) => console.log("SSE data received", e.data);
        }
    }

I pulled in the pushpin image and set it up with the following manifests:

apiVersion: v1
kind: ConfigMap
metadata:
  name: pushpin
  namespace: {{ .Values.namespace }}
data:
  routes: |
    *,debug nrs:8800 //our app
---
apiVersion: v1
kind: Service
metadata:
  labels:
    mm.service: pushpin
  name: pushpin
  namespace:  {{ .Values.namespace }}
spec:
  ports:
    - name: pushpin
      port: 7999
      targetPort: 7999
  selector:
    mm.service: pushpin
---
apiVersion: apps/v1
kind: ReplicaSet
metadata:
  creationTimestamp: null
  labels:
    mm.service: pushpin
  name: pushpin
  namespace: {{ .Values.namespace }}
spec:
  replicas: 1
  selector:
    matchLabels:
      mm.service: pushpin
  template:
    metadata:
      creationTimestamp: null
      labels:
        mm.service: pushpin
    spec:
      containers:
        - image: fanout/pushpin
          name: pushpin
          ports:
            - containerPort: 7999
          volumeMounts:
          - mountPath: /etc/pushpin/routes
            name: nrs-pushpin
            subPath: routes
      hostname: pushpin
      restartPolicy: Always
      volumes:
        - name: nrs-pushpin
          configMap:
            name: pushpin
            items:
            - key: routes
              path: routes
---

When I access this page, I get the initial 'SSE connection made' with a connection event, but then I never see 'SSE data received', even though the logutility is routinely receiving messages and calling send_event.

The way this is setup in the cluster is I have an ingress point that routes traffic to my frontend service (after validating through keycloak), and that initial stuff all seems to be fine. It's just somewhere the send_event in Django isn't getting back to my client. Not sure if it's something as simple as a channel misnaming issue or what.

The pushpin logs have the following when I initially connect:

[INFO] 2023-04-10 23:11:02.796 [zurl] IN id=f5c16c86-35e0-43ce-a717-08dc2c3a9ef7, GET http://pushpin:7999/event/logmessage
[INFO] 2023-04-10 23:11:02.806 [zurl] OUT id=f5c16c86-35e0-43ce-a717-08dc2c3a9ef7 code=301 0
[INFO] 2023-04-10 23:11:02.810 [proxy] GET http://pushpin:7999/event/logmessage -> nrs:8800 ref=https://nrs./ code=301 0
[INFO] 2023-04-10 23:11:02.885 [zurl] IN id=fbb1f830-41c7-49cd-a7ab-59969d0c95f9, GET http://pushpin:7999/event/logmessage/
[INFO] 2023-04-10 23:11:02.891 [zurl] OUT id=fbb1f830-41c7-49cd-a7ab-59969d0c95f9 code=200 2077
[INFO] 2023-04-10 23:11:02.910 [handler] subscribe http://pushpin:7999/event/logmessage/ channel=events-logmessage
[INFO] 2023-04-10 23:11:02.910 [handler] subscribe http://pushpin:7999/event/logmessage/ channel=user-anonymous
[INFO] 2023-04-10 23:11:02.914 [proxy] GET http://pushpin:7999/event/logmessage/ -> nrs:8800 ref=https://nrs./ accept
[INFO] 2023-04-10 23:11:03.993 [zurl] IN id=49c61ed3-2bfd-4400-a6f4-e63a3e29f652, GET http://pushpin:7999/event/logmessage/?link=next&recover=true&es-meta=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJlcyIsImV4cCI6MTY4MTE3MTg2MiwiY2hhbm5lbHMiOlsibG9nbWVzc2FnZSJdLCJ1c2VyIjoiYW5vbnltb3VzIn0.gfBVwH7APFGfxJqOHERTwlHdWo-5vrxwrWet1R2iFHk
[INFO] 2023-04-10 23:11:04.005 [zurl] OUT id=49c61ed3-2bfd-4400-a6f4-e63a3e29f652 code=200 0
[INFO] 2023-04-10 23:11:04.007 [handler] GET http://pushpin:7999/event/logmessage/?link=next&recover=true&es-meta=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJlcyIsImV4cCI6MTY4MTE3MTg2MiwiY2hhbm5lbHMiOlsibG9nbWVzc2FnZSJdLCJ1c2VyIjoiYW5vbnltb3VzIn0.gfBVwH7APFGfxJqOHERTwlHdWo-5vrxwrWet1R2iFHk route=*,*,*,* code=200 0

Any help here would be appreciated, not sure where to try and debug the connection or how to even test in this setup if Pushpin/django-eventstreams is trying to send the messages. I know there are channel selectors that I will eventually simplify my urls path with (since I have several channels I will eventually be sending on), but wanted to start out straight forward with all this but unfortunately that's not proving as easy as I hoped.

Thanks!

Getting django-eventstream and Pushpin working

Answers (1)

Related Questions