CODEWITHSUNDEEP
javascriptaws-sdkamazon-sqs
Claudiordgz
Claudiordgz

Reputation: 3049

Size of SQS messages

I'm sending messages to SQS using the AWS-SDK for JavaScript. Each message needs to be 256kb in size tops.

Each message is a JSON object that gets decoded on another service.

Option 1: JSON Object as string: Count length and make sure it's less than 262144?

function* getStuff(rows, someConfig) {
  let totesPayload = 0
  let payload = []
  for (const row of rows) {
   const singleItemInPayload = rowToPayload(row, someConfig)
   if (singleItemInPayload.length + totesPayload < 262144 - (enclosingObjectSize())) {
     payload.push(singleItemInPayload)
     totesPayload += singleItemInPayload.length
   } else {
     yield({ payload })
     payload = []
   }
}

Option 2: Buffer.from(JSON Object as string): Count length of JSON Object and make sure it's less than 262144?

Most of the data is text, so I'm not sure I'm going to get any good help from putting it in a byte array.

Is option 2 necessary?

Upvotes: 2

Views: 2828

Answers (1)

Matthew Pope
Matthew Pope

Reputation: 7669

SQS uses UTF-8 for strings, so if any part of your message can contain non-ASCII characters, then you will need to measure the size by converting to bytes because UTF-8 is a variable width encoding which uses 1 to 4 bytes for a single character.

Upvotes: 3

Related Questions