CODEWITHSUNDEEP
mongodbamazon-web-servicesamazon-dynamodb
Jack
Jack

Reputation: 121

MongoDB to DynamoDB

I have a database currently in Mongo running on an EC2 instance and would like to migrate the data to DynamoDB. Is this possible and what is the most cost effective way to achieve this?

Upvotes: 9

Views: 12108

Answers (3)

Guru
Guru

Reputation: 84

I know this post is old, Amazon made it possible with AWS DMS, check this document :

https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.MongoDB.html

Some relevant parts:

Using an Amazon DynamoDB Database as a Target for AWS Database Migration Service

You can use AWS DMS to migrate data to an Amazon DynamoDB table. Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. AWS DMS supports using a relational database or MongoDB as a source.

Upvotes: 1

Pete Garafano
Pete Garafano

Reputation: 4913

Whenever you are changing databases, you have to be very careful about the way you migrate data. Certain data formats maintain type consistency, while others do not.

Then there are just data formats that cannot handle your schema. For example, CSV is great at handling data when it is one row per entry, but how do you render an embedded array in CSV? It really isn't possible, JSON is good at this, but JSON has its own problems.

The easiest example of this is JSON and DateTime. JSON does not have a specification for storing DateTime values, they can end up as ISO8601 dates, or perhaps UNIX Epoch Timestamps, or really anything a developer can dream up. What about Longs, Doubles, Ints? JSON doesn't discriminate, it makes them all strings, which can cause loss of precision if not deserialized correctly.

This makes it very important that you choose the appropriate translation medium. The generally means you have to roll your own solution. This means loading up the drivers for both databases, reading an entry from one, translating, and writing to this other. This is the best way to be absolutely sure errors are handled properly for your environment, that types are kept consistently, and that the code properly translates schema from source to destination (if necessary).

What does this all mean for you? It means a lot of leg work for you. It is possible somebody has already rolled something that is broad enough for your case, but I have found in the past that it is best for you to do it yourself.

Upvotes: 2

ketan vijayvargiya
ketan vijayvargiya

Reputation: 5649

When you ask for a "cost effective way" to migrate data, I assume you are looking for existing technologies that can ease your life. If so, you could do the following:

  • Export your MongoDB data to a text file, say in tsv format, using mongoexport.
  • Upload that file somewhere in S3.
  • Import this data, in S3, to DynamoDB using AWS Data Pipeline.

Of course, you should design & finalize your DynamoDB table schema before doing all this.

Upvotes: 6

Related Questions