CODEWITHSUNDEEP
amazon-s3amazon-ec2amazon-web-servicescloudamazon-ebs
user476566
user476566

Reputation: 1349

S3 or EBS for storing data in flat files

I have flat files in which I store data and retrieve it instead of storing to database. This is temporary and may last for couple of months.I was wondering If I should be using EBS or S3. EBS is mainly used for I/O , S3 for content delivery , but S3 is on use you go model and EBS is you have to pay for the volume purchased ? Pls guide, which one is better ?

Upvotes: 0

Views: 3167

Answers (4)

Harish Ganesan
Harish Ganesan

Reputation: 743

Point 1) You can use both S3 and EBS for this option. If you want reduced latency and file sizes are bigger then EBS is better option.

Point 2) If you want lower costs, then S3 is a better option.

Upvotes: 1

Chris M.
Chris M.

Reputation: 1851

From what you describe, S3 will be the most cost-effective and likely easiest solution.

Pros to S3: 1. You can access the data from anywhere. You don't need to spin up an EC2 instance. 2. Crazy data durability numbers. 3. Nice versioning story around buckets. 4. Cheaper than EBS

Pros to EBS 1. Handy to have the data on a file system in EC2. That let you do normal processing with the Unix pipeline. 2. Random Access patterns work as you would expect. 3. It's a drive. Everyone knows how to deal with files on drives.

If you want to get away from a flat file, DynamoDB provides a nice set of interfaces for putting lots and lots of rows into a table, then running operations against those rows.

Upvotes: 0

Mike Brant
Mike Brant

Reputation: 71422

To build on the good answer from willglynn. If you are interacting with the data regularly, or need more file-system-like access you might consider EBS more strongly.

If the amount of data is relatively small and you read and write to the data store regularly, you might consider something like elasticache for in-memory storage which would likely be superior performance-wise then using s3 or EBS.

Similarly, you might look at DynamoDb for document type storage, especially if you need to be able to search/filter across your data objects.

Upvotes: 1

willglynn
willglynn

Reputation: 11520

S3 sounds like it's more appropriate for your use case.

S3 is object storage. Think of it as an Amazon-run file server. (Objects are not exactly equal to files, but it's close enough here.) You tell S3 to put a file, it'll store it. You tell S3 to get a file, it'll get return it. You tell S3 to delete it, it's gone. This is easy to work with and very scalable.

EBS is block storage. Think of it as an Amazon-run external hard drive. You can plug an EBS volume into an EC2 virtual machine, or you access it over the Internet via AWS Storage Gateway. Like an external hard drive, you can only plug it into one computer at a time. The size is set up front, and while there are ways to grow and shrink it, you're paying for all the bits all the time. It's also much more complex than S3, since it has to provide strong consistency guarantees for the entire volume, instead of just on a file-by-file basis.

Upvotes: 4

Related Questions