CODEWITHSUNDEEP
androidiosamazon-ec2rabbitmqamqp
berardo
berardo

Reputation: 25

Amazon EC2 RabbitMQ cluster costs

Sorry guys, but I tried to find out this question but I didn't get exactly what I need.

I'm developing an Android/iPhone app that will connect to a RabbitMQ cluster at Amazon's Ubuntu EC2.

I need to calculate how much it will be. I believe that it will have ~100k users. I'm configuring the broker to have one user and one queue per mobile endpoint which will be connected over all the time (with basicConsume and manual ack - I believe that I wont need transactions). These users and queues will be created by an unique PHP server using the REST RabbitMQ port. The mobile endpoints will open another connection to basicPublish messages to one of my only 5 topic exchanges. This exchanges will need to deliver the message up to ~2k queues. Each mobile endpoint will consume about 20 messages a day (produced by other endpoint, so I don't know exactly if it will be 20, 2 or 2k messages a day ;) . Each message body will be only what the user usually types. I am using AMQP headers and parameters to all metadata.

I've read about resource usage at an old Nabble post and I'm trying to calculate: Just to keep one idle connection the server needs ~66kb and 8 processes (btw, I've changed heartbeat down to 60s), so:

100k connections * 66kb = 6.29Gb 100k connections * 8 processes = 800k processes (is it possible for a unique machine?)

I don't know yet if I will need to persist all the messages. At this time, I'm thinking don't persist.

Here in my madness I'm guessing that two "Linux on m3.xlarge EBS Optimized" (4 vCPU and 15Gb memory each, high I/O) in a cluster could be suitable to this project.

Am I wrong?

In this scenario, what would you use and how would you calculate? Any tips to figure out the best architecture that suits this projects or item to don't forget at Amazon's calculator will be gold for me.

Thanks in advance.

Upvotes: 0

Views: 524

Answers (1)

Karell Ste-Marie
Karell Ste-Marie

Reputation: 1052

No, you are not wrong by much

I have a few question but essentially it boils down to:

  1. What do you know about your application?
  2. What does an individual user need?

The only way to understand your scenario is to properly understand your most demanding user and seeing what their resource requirements are.

After that... multiply

Do you know what the CPU requirements for all those queues and all those messages are? Did you try to perform a load test for say 24 hours on several different instances? Then all you have to do is calculate the cost/hr of the instances.

When we (our company) want to measure the necessary resources for a project we essentially load we will essentially write a system that will simulate the load on the server instances and push it to the maximum - it then gives us a picture of what will be required instance size, etc. With the AWS tools you can then easily get a clear picture of I/O, Bandwidth, etc.

Hope this helps

Upvotes: 0

Related Questions