mongoDB architecture for scalable read-heavy app (constant writes)

Question

My app runs a daily job that collects data and feeds it to a mongoDB. This data is processed and then exposed via rest API.

Need to setup a mongodb cluster in AWS, the requirements:

• Data will grow about the same size each day ( about 50M records), so write throughput doesn't need to scale. writes would be triggered by a cron at a certain hour. Objects are immutable ( they won't grow)
• Read throughput will depend on number of users / traffic, so it should be scalable. traffic won't be heavy in the beginning.
• Data is mostly simple JSON, need a couple of indices around some of the fields for fast-querying / filtering.

what kind of architecture should I use in terms of replica sets, shards, etc ?.

What kind of storage volumes should I use for this architecture? ( EBS, NVMe) ? Is it preferred to use more instances or to use RAID setups. ?

I'm looking to spend some ~500 a month.

Thanks in advance

Answer 1

To setup the MongoDB cluster in AWS I would recommend to refer the latest AWS quick start for MongoDB which will cover the architectural aspects and also provides CloudFormation templates.

For the storage volumes you need to use EC2 instance types that supports EBS instead of NVMe storage since NVMe is only an instance storage. If you stop and start the EC2, the data in NVMe is lost.

Also for the storage volume throughput, you can start with General Purpose IOPS with resonable storage size and if you find any limitations then only consider Provisioned IOPS.

For high availability and fault tolerance the CloudFormation will create multiple instances(Nodes) in MongoDB cluster.