CODEWITHSUNDEEP
pythonpython-3.xflaskwebsocketpython-asyncio
alt-f4
alt-f4

Reputation: 2326

How can I serve stored data from web-sockets using Flask?

TLDR; I am trying to run a client that fetches data from a web-sockets URI, and then use Flask to serve the fetched data from the sockets.

My intended workflow looks like: Read data from web-sockets (async) -> write data to a Python dict (continuously) -> read data from Python dict using GET requests on Flask (continuously)

The problem I am facing is that I am using a Python dictionary for storage, but when I read that dictionary from Flask, it does not show me updated values.

I have created an illustration with code to my problem

Client:

import asyncio
import websockets
import json

SOME_URI = "ws://localhost:8080/foo"
connections = set()
connections.add(SOME_URI)


class Storage:
    storage = dict() # local storage dict for simplicity

    @staticmethod
    def store(data): # here I store the value
        a, b, c = data
        Storage.storage[a] = c

    @staticmethod
    def show(): # Here I just show the value to be used in the GET
        return Storage.storage


async def consumer_handler(uri):
    async with websockets.connect(uri) as websocket:
        async for message in websocket:
            await consumer(message)


async def consumer(message):
    line = json.loads(message)
    Storage.store(line) # adds message to dict


async def main():
    await asyncio.wait([consumer_handler(uri) for uri in connections])


if __name__ == "__main__":
    asyncio.run(main())

App:

from flask import Flask
from client import Storage

app = Flask(__name__)
app.debug = True


@app.route('/bar', methods=['GET'])
def get_instruments():
    res = Storage.show() # I expected here to see updated value for the dict as it fills up from websockets
    return res, 200


if __name__ == "__main__":
    app.run()

Whenever I try to make a GET request to the Flask page, I get the value of an Empty dict (does not reflect the changes I add from the web socket). I was expecting to get an updated value of the dict with every GET request.

Upvotes: 1

Views: 617

Answers (1)

heemayl
heemayl

Reputation: 42097

The problem is that you have two separate Python processes involved here -- one is the async script for ingesting data and the other is running flask. As these are separate processes, the dict updates that you're doing on the first process is not visible to the second process.

You need some form of IPC (Inter-Process Communication) mechanism here for data portability and/or persistence. One obvious and easier option to try is to use a named FIFO or a plain file. Depending on the level of complexity you want to handle for this, using Redis (or alike) is also another option.

Upvotes: 1

Related Questions