AWS S3 alternatives for private cloud

Question

Right now we have a requirement to migrate from AWS to private Data Center. We need to find out potential alternative storage instead of AWS S3. Currently S3 is used in the following way:

• Overall storage size is 10TB;
• Min/Avg/Max object size is 0.5/2/100 Mb;
• We have N App instances that simultaneously writes/reads
  objects approximately 50 writes/sec, 30 reads/sec;
• This storage should be redundant (Highly Available), Fault Tolerant, Scalable;

The naive implementation could be store this data on:

• Simple NFS storage and add some replication functionality;
• Just store mentioned objects in NoSQL DB (as example in Cassandra). However Cassandra will require a number of instances to support this storage (It's nor recommended to store > 1TB pn 1 Cassandra node Cassandra capacity planning)

What solution would you recommend for such scenario ?

Answer 1

Using MinIO, an object store built for AI storage, is your best bet if you want to have a private cloud storage. It is AWS S3 compatible meaning that applications use AWS S3 can be migrated to MinIO seamlessly. They have a tutorial how to connect MinIO server with AWS CLI. You can test it against the public hosted MinIO server https://play.min.io:9000. Please refer to AWS CLI with MinIO Server.

You can have highly available storage system using MinIO distributed setup. Beware that the dynamic expansion is not a feature of MinIO distributed setup. If you want to expand your cluster you end up spinning a new cluster with your desired number of servers/disks and then you have to migrate your data from old one to new one.

I find it much more easier to use than HDFS. In addition to this, there are a lot of technologies outside Hadoop ecosystem lack HDFS integration. For example, Docker Registry lacks built in HDFS storage driver. However, it has a S3 driver so you can use MinIO as it's object storage.

Answer 2

There're a bunch of options as of S3-compatible private cloud service. if you like open source solutions, the above open stack and Cassandra are good ones. Note that usually no matter what you use, probably you end up setting up a cloud with multiple nodes and this is inevitable to exchange for redundancy and availability. There're some good commercial and economic products as well such as the one from Cloudian

Answer 3

You can use OpenStack Swift

Swift is a highly available, distributed, eventually consistent object/blob store. Organizations can use Swift to store lots of data efficiently, safely, and cheaply.

Learn More on : https://docs.openstack.org/swift/latest/
And https://oldhenhut.com/2016/05/31/s3-vs-swift/

Answer 4

If you need object store I could recommend elliptics (in english). As I know, it doesn't has limits on disk store.

In case for Cassandra we are using SSD disks (for better performance) < 200-500 Gb. Ring size would be depend from your requirements (read/write latency, replication rate, time to life).

50 writes/sec, 30 reads/sec

This is really quite easy for Cassandra, as I can compare with our setup. In that case it more depends from time to life for your objects.

Generally, in case for distributed network you also could look at GlusterFS.