CODEWITHSUNDEEP
amazon-web-servicesamazon-s3redisaws-lambdaamazon-dynamodb
Nir-Z
Nir-Z

Reputation: 869

AWS - Best way to transfer and retreive large scale data from/to AWS storage

I have an Ionic mobile hybrid app that use AWS in the middle like API gateway trigerring lamdba functions for simple post, get requests.

I want to store large jsons (arround 5K items) on AWS so they could be easily retreived and stored in the application's local storage.

I'm planning to create a ETL cron job that will upload the data from my server to the AWS for this purpose

AWS offers few services like Redis, DynamoDB that can be used for this scenario but I'm confused about the method this could be acheived.

I thought and read about few options:

  1. API gateway -> DynamoDB
  2. API gateway -> lambda function -> DynamoDB
  3. API gateway -> lambda function -> Redis

I also read about S3 and pipline, but I'm not sure that they would fit.

Please advise.

Thanks

Upvotes: 1

Views: 689

Answers (2)

qkhanhpro
qkhanhpro

Reputation: 5220

If I understand your question correctly, you want to put JSON files data from your server into AWS so that your mobile client ( using Lambda, etc... ) have faster access to that data

In that case, AWS S3 should suffice ( providing that you know what JSON file will be served to your client and you don't have to look into / query it's content ). It should look like so :

[ Server ] ==[Whatever]==> [ AWS S3 ]

[ AWS S3 ] <==[Lambda]==> [ Mobile client ]

For that [ Whatever ] , take a look at Amazon's document. May be sync can help

Edit : After reading your comment, I realize your intention is to put data currently on your oracle DB into a JSON based DB on AWS. In that case, DynamoDB is the way to go. You should follow @Haresh Chhelana 's solution and edit your question to be more clear

You can also leverage DMS with DynamoDB as the target if you don't need advanced data transformation before migrating to DynamoDB

Upvotes: 1

Hareshkumar Chhelana
Hareshkumar Chhelana

Reputation: 24848

First of all, Redis is not best fit for your requirement. Using API Gateway with DynamoDB or Lambda will cover your requirement.

1. API Gateway -> DynamoDB: Choose if you just try to store JSON data into DynamoDB without

2. API Gateway -> Lambda -> DynamoDB: Choose if you required performing some operation or modification on JSON data before storing into DynamoDB.

Suggestion: Choose the first approach (API Gateway -> DynamoDB) for store JSON data into DynamoDB using API Gateway POST request and retrieve JSON data based on the query using API Gateway GET request.

How you can implement the first approach follow this Amazon blog: https://aws.amazon.com/blogs/compute/using-amazon-api-gateway-as-a-proxy-for-dynamodb/

Upvotes: 1

Related Questions