CODEWITHSUNDEEP
amazon-web-servicesamazon-s3storageminio
GreenLake4964
GreenLake4964

Reputation: 776

Context of using Object Storage

Can someone please give me a solid use case example (in terms of performance/scalability/reliability) that an Object Storage FS should be used over Block Storage FS?

I am so confused since PM wants us to use MinIO in a company's project, but didn't give reasons for why are we should use it without any obvious pros?

I have read below posts, but I don't think these resolve my query.
What is Object storage really?
Difference between Object Storage And File Storage

Upvotes: 0

Views: 132

Answers (1)

John Rotenstein
John Rotenstein

Reputation: 269971

Block storage is a traditional disk. In the old days, you would walk down to the computer store, buy a Hard Disk Drive that has a specific amount of storage (eg 1TB), plug it into the computer and then use it for storage. If it runs out of disk space, you had to delete files or buy additional disks.

This type of storage is called 'Block Storage' because the disk is divided into many blocks. The Operating System is responsible for managing what data is stored in each block and it also maintains a directory of those files. You'll see terms like FAT32 or exFAT for these storage methods.

Computers expect to have this type of directly attached disk storage. It's where the computer keeps its operating system, applications, files, etc. It's the C: and D: drives you see on Windows computers. When using services like Amazon EC2, you can attach Block Storage by using the Amazon Elastic Block Store service (Amazon EBS). Even though storage is virtual (meaning you don't need to worry about the physical disks), you still need to specify the size of the disk because it is pretending to be a traditional disk drive. Therefore, you can run out of space on these disks (but it is fairly easy to expand their size).

Next comes network-attached storage. This is the type of storage that companies give employees where they can save their documents on the network instead of a local disk (eg the H: drive). The beauty of network-attached storage is that it doesn't look like blocks on a disk -- instead, the computer just says "save this file" or "open this file". The request goes across the network to a file server, which is responsible for storage the actual data on the disk. This is a much more efficient way to store data since it is centralized rather than on everybody's computer and it is much easier to backup. However, your company still needs to have disk drives that store the actual data.

Then comes object storage which has become popular with the Cloud. You can store files in Amazon S3 (or MinIO, which is S3-compatible) without worrying about hard disks and backups -- they are the job of the 'cloud'. You simply store data and somebody else worries about how that data is stored. It is typically charged via pay-as-you-go, so instead of buying expensive hard disks up-front, you just pay for the amount of storage used. The data is typically automatically replicated between multiple disks so it can survive the failure of disk drives and even data centers. You can think of cloud-based block storage as unlimited in size. (It isn't actually unlimited, but it acts like it is.)

Services like S3 and MinIO also do more than simply store the data. They can make the objects available via the Internet without having to run a web server. They can have fine-grained permissions to control who (and what) can access the data. Amazon S3 is very 'close' to other AWS services, making it very fast to use data in Amazon EC2, Amazon EMR, Amazon RDS, etc. It's even possible to use query engines like Amazon Athena that allow you to run SQL commands to query data stored in Amazon S3 without needing to load it into a database. You can choose different storage classes to reduce cost while trading-off access speed (like the old days of tape backup). So think of Object Storage as 'intelligent storage' that is much more capable than a dumb disk drive.

Bottom line: Computers expect to have block storage to boot and run apps, but block storage isn't a great way to manage the storage of large amounts of data. Object Storage in the Cloud is as simple as uploading and downloading data without having to worry about how it is stored and how to manage it -- that is the cloud's job. You get to spend your time adding value to your company rather than managing storage on disk drives.

Upvotes: 1

Related Questions