CODEWITHSUNDEEP
amazon-web-servicesamazon-dynamodbamazon-sqs
Shashi
Shashi

Reputation: 315

Insert data inside SQS queue in order into DynamoDB

Can we have multiple lines in an sqs message and insert them in order to dynamodb? for example if i have multiple sqs queues by departments and inside queue i have list of employees which have attributes like employee id, dept id, first name, last name and i should iterate the contents of queue in ascending order of employee id and insert employee id, dept id, first name, last name into dynamodb table.

Upvotes: 0

Views: 709

Answers (1)

lovelywib
lovelywib

Reputation: 579

From what I read, it looks like you want the employee data kept sorted by id in Dynamodb.

First, SQS doesn't do sorting on your demand/customization. It does provide FIFO (first in first out) queues which are based on the order you insert the messages.

Even if SQS does have the feature you wanted and you can insert message/data to dynamodb in your preferred order, it doesn't gurantee you having the data sorted in dynamodb and sorted in your query results.

In order to have data sorted in DynamoDB table and reflected on your query results, you'll need to use sort key (range attribute).

Here are few options you can try on DynamoDB table design.

  1. dept id as partition key + employee id as sort key

With this setup, you have your employee data sorted within one department when you do queries.

  1. create an attribute/column named "everyone", set value to 1 when inserting the records. Then you can create GSI (Global Secondary Index) with everyone as partition key and employeeId as sort key.

With this setup, you have your all employee data sorted by employee ids, regardless department id. It should work well if your GSI size is below a few gigabytes.

Also, you can combine option 1 and option 2 together to archive sorting employees both on campany level and department level.

Upvotes: 1

Related Questions