Sending binary file through RabbitMQ

Question

I'm spec'ing a design right now that uses RabbitMQ as a message queue. The message are going to have a JSON body, and for one message in particular I'd like to add a small binary file.

What I'd like to know is, should the binary file's data be part of the JSON message, or can it be appended to the message separately?

Answer 1

We use RabbitMQ and file transfer. It's working a little slow, but for updating far away customers it is duable. I would recommend you following guidelines: Create a message structure for each block you send with command, offset and CRC32, data length, maximum 60kByte per block of data, give blocks a counter, do a sha256 at the end to make sure data is correct, make tar optional so the data can be much smaller and transmission is faster.. make a directory thing, to see which files need to update... use a broadcast event to see who's available and update 1 customer at a time with client.company.update orso, where a customer listens to .company. have fun! p.s. we also created a linux-service for this, which starts automatic at the customer.

Answer 2

Since RabbitMQ message payload is just a binary array you should encode your message body with 3 fields:

1. File size
2. Binary data of a file
3. Json

I disagree with a previous answer about embedding a file in json. If you encode file data inside of json you will get wasted space because of json escaping + unnecessary CPU usage because of json encoding/decoding of the file data + you will need to read file data twice (once for json deserialization and once more to copy it where it needs to go)

Answer 3

The message is a single byte stream, it also contains a header but that is less relevant. I suggest that you take one of two approaches. Your JSON object contains a field that is the byte stream of the binary file. Alternatively it gives the address where the file can be downloaded. If the binary file is small then the former should work fine. Otherwise you may prefer the latter.