CODEWITHSUNDEEP
pythonamazon-web-servicesamazon-ec2botoamazon-sqs
Parzival
Parzival

Reputation: 2064

Amazon SQS messages with arbitrary Python objects?

So, I'm trying to use SQS to pass a Python object between two EC2 instances. Here's my failed attempt:

import boto.sqs
from boto.sqs.message import Message

class UserInput(Message):
    def set_email(self, email):
        self.email = email
    def set_data(self, data):
        self.data = data
    def get_email(self):
        return self.email
    def get_data(self):
        return self.data

conn = boto.sqs.connect_to_region('us-west-2')
q = conn.create_queue('user_input_queue')
q.set_message_class(UserInput)
m = UserInput()
m.set_email('[email protected]')
m.set_data({'foo': 'bar'})
q.write(m)

It returns an error message saying that The request must contain the parameter MessageBody. Indeed, the tutorial tells us to do m.set_body('something') before writing the message to the queue. But here I'm not passing a string, I want to pass an instance of my UserInput class. So, what should MessageBody be? I've read the docs and they say that

The constructor for the Message class must accept a keyword parameter “body” which represents the content or body of the message. The format of this parameter will depend on the behavior of the particular Message subclass. For example, if the Message subclass provides dictionary-like behavior to the user the body passed to the constructor should be a dict-like object that can be used to populate the initial state of the message.

I guess the answer to my question might be in that paragraph, but I can't make sense of it. Could anyone provide a concrete code example to illustrate what they are talking about?

Upvotes: 4

Views: 2995

Answers (2)

Erlandsen-tech
Erlandsen-tech

Reputation: 74

You can use pickle and base64 to do this. To clarify:

import pickle
import base64
import boto3

sqs = boto3.client('sqs')
arbitrary_object = { "test_arbitrary_object": "content of variable"}
message_body =str(base64.b64encode(pickle.dumps(arbitrary_object)))
            self.log.info("Sending transactions")
            sqs.send_message(QueueUrl=queue_url, MessageBody=message_body)

To unpickle you do the opposite on the receiving end. Base64 decode and pickle.load. I hope this helps to solve your issue.

Upvotes: 0

Simeon Visser
Simeon Visser

Reputation: 122326

For an arbitrary Python object the answer is to serialize the object into a string, use SQS to send that string to the other EC2 instance and deserialize the string back to an instance of the same class.

For example, you could use JSON with base64 encoding to serialize an object into a string and that would be the body of your message.

Upvotes: 5

Related Questions