CODEWITHSUNDEEP
erlang
Duke
Duke

Reputation: 21

How to scale out Erlang application to utilize system resource efficiently?

I am making an application with many users.

However, I am getting stuck with "scale out to utilize exhaustedly system resource." I hope you understand what I mean by that part in quotes. If not, maybe I can clarify in the future.

The application supports SCTP traffic with around 50k messages/sec.

I am using 10 supervisors with a lot of processes are under as workers.

Some distributors are used to transfer SCTP messages to the workers get by process id.

The distributors work well with no delay, no drop any message.

The workers have to do some tasks so I can see the distributors' message queue length increasing up.

But, the workers and supervisors are "getting hang on." :S

I can capture this situation via Observer with all scheduler threads are busy (red color 100%) (I am using chipset with 32 cores and 64 GB RAM).

Meanwhile, CPU load and RAM consumption are not high (using htop command).

So could you let me know how to turn up this application to utilize almost system resource?

Seems there are some bottleneck in OTP 18?

Thanks and regards, Duke.

Upvotes: 1

Views: 89

Answers (1)

Duke
Duke

Reputation: 21

Thanks for your time to look into this post. Gotcha !!! I found the problem.It's belong to Mnesia read/write performance (I missed to mention that I am using Mnesia for DB management). I read many threads discussing about Mnesia performance. With my experience, can say: Mnesia with fragment will give us a best performance, much more than mnesia:ets or mnesia:async_dirty.

Cheers, Duke

Upvotes: 1

Related Questions