CODEWITHSUNDEEP
pythonmqttpahosubscribe
Tranks Naing
Tranks Naing

Reputation: 95

How to handle partial messages in MQTT subscriber?

I have an IoT gateway device that publishes data to a MQTT broker every minute. However, the message comes as several packets. I need to recombine these partial message packets into one complete message, parse it and then store in a database. I'm using paho python MQTT library. The problem is that the callback is triggered upon each received partial packet but I want to trigger it only when all packets associated with the same message are received. Is there anyway to do that? Cheers

Sample partial messages look like this:

Packet 1 =

{'DB3': {'CH1-01': '243.7300', 'CH1-02': '0.0000', 'CH1-03': '0.0000', 'CH1-04': '243.7300', 'CH1-05': '0.0000', 'CH1-06': '243.7300', 'CH1-07': '0.0000', 'CH1-08': '0.0000', 'CH1-09': '0.0000', 'CH1-10': '0.0440', 'CH1-11': '0.0000', 'CH1-12': '0.0000', 'CH1-13': '0.0440', 'CH1-14': '-0.0040', 'CH1-15': '0.0000', 'CH1-16': '0.0000', 'CH1-17': '-0.0040', 'CH1-18': '0.0440', 'CH1-19': '0.0000', 'CH1-20': '0.0000', 'CH1-21': '0.0440', 'CH1-22': '0.0000', 'CH1-23': '0.0000', 'CH1-24': '0.0000', 'CH1-25': '0.0000', 'CH1-26': '50.0160', 'CH1-27': '3.6400', 'CH1-28': '0.0000', 'CH1-29': '0.0000', 'CH1-30': '0.4000', 'CH1-31': '3.6400', 'CH1-32': '0.0000', 'CH1-33': '0.0000', 'CH1-34': '0.4000', 'CH1-35': '0.0000', 'CH1-36': '0.0000', 'CH1-37': '0.0000', 'CH1-38': '0.0000', 'CH1-39': '0.0000', 'CH1-40': '0.0000', 'CH1-41': '0.0000', 'CH1-42': '0.0000', 'CH2-01': '243.6100', 'CH2-02': '0.0000', 'CH2-03': '0.0000', 'CH2-04': '243.6100', 'CH2-05': '0.0000', 'CH2-06': '243.6100', 'CH2-07': '0.0000', 'CH2-08': '0.0000'}, 'ts': 1637040150000}

Packet 2 =

{'DB3': {'CH2-09': '0.0000', 'CH2-10': '0.0000', 'CH2-11': '0.0000', 'CH2-12': '0.0000', 'CH2-13': '0.0000', 'CH2-14': '0.0000', 'CH2-15': '0.0000', 'CH2-16': '0.0000', 'CH2-17': '0.0000', 'CH2-18': '0.0040', 'CH2-19': '0.0000', 'CH2-20': '0.0000', 'CH2-21': '0.0040', 'CH2-22': '0.0000', 'CH2-23': '0.0000', 'CH2-24': '0.0000', 'CH2-25': '0.0000', 'CH2-26': '50.0200', 'CH2-27': '0.0000', 'CH2-28': '0.0000', 'CH2-29': '0.0000', 'CH2-30': '0.0000', 'CH2-31': '0.0000', 'CH2-32': '0.0000', 'CH2-33': '0.0000', 'CH2-34': '0.0000', 'CH2-35': '0.0000', 'CH2-36': '0.0000', 'CH2-37': '0.0000', 'CH2-38': '0.0000', 'CH2-39': '0.0000', 'CH2-40': '0.0000', 'CH2-41': '0.0000', 'CH2-42': '0.0000', 'CH3-01': '243.6400', 'CH3-02': '0.0000', 'CH3-03': '0.0000', 'CH3-04': '243.6400', 'CH3-05': '0.0000', 'CH3-06': '243.6400', 'CH3-07': '0.0000', 'CH3-08': '0.0000', 'CH3-09': '0.0000', 'CH3-10': '0.0000', 'CH3-11': '0.0000', 'CH3-12': '0.0000', 'CH3-13': '0.0000', 'CH3-14': '0.0000', 'CH3-15': '0.0000', 'CH3-16': '0.0000'}, 'ts': 1637040150000}

Upvotes: 0

Views: 238

Answers (1)

hardillb
hardillb

Reputation: 59866

This isn't a really a MQTT or Paho question, both are working exactly as expected, the problem is with the device publishing the messages if there is no way to determine how many parts it will send or the start/end of a stream of packets.

My best suggestion is to set a timeout after the first message and when that expires assemble all the messages that arrived in that period. What you set that timeout to you'll have to determine, but if you have configured the data to come in bursts every 60 seconds then possibly start at 15 seconds.

Upvotes: 1

Related Questions