CODEWITHSUNDEEP
pythondjangowebsocketdjango-channels
Jack022
Jack022

Reputation: 1287

How much data can a websocket consumer handle?

I've built a simple application using Django Channels where a Consumer receives data from an external service, the consumer than sends this data to some subscribers on another consumer. I'm new to websockets and i had a little concern: the consumer is receiving a lot of data, in the order of 100 (or more) JSON records per second. At what point should i be worried about this service crashing or running into performance issues? Is there some sort of limit for what i'm doing?

Upvotes: 0

Views: 4488

Answers (1)

Matthaus Woolard
Matthaus Woolard

Reputation: 2408

there is not explicit limit, however it is worth nothing that for each instances (open connection) of the consumer you can only process one WS message at once.

So if you have a single websocket connection and are sending lots and lots of WS messages down that connection if the consumer does work on these (eg write them to the db) the queue of messages might fill up and you will get an error.

their are a few solutions to this,

  1. Open multiple ws connections and share out the load

  2. in your consumer before doing any work that will take time put it onto a work queue and have some background tasks (that you do not await) consume it.

For this second option it is probably a good idea to create this background queue in your on_connect method and handle shutting it down/waiting for it to flush everything in the on disconnect method.

--

if you are expecting a massive amount of data and don't want to fork out for a costly (high memory) VM you might be better of using a server that is no written in python.

My suggestion for a python developer would be https://docs.vapor.codes/4.0/websockets/ this is a Swift server framework, Swift linguistically if very close to TypeAnotated python so is easier than other high performance options to pick up for a python dev.

Upvotes: 2

Related Questions