openmobster
openmobster

Reputation: 1027

Netty's ability to handle large payloads

I am using Apache MINA in my open source project. However, I have hit a performance road block where it is not being able to handle large payloads uploaded to the server from mobile devices. I am getting Out of Memory errors. I have never used Netty and looking to swap MINA with Netty.

My question for the Netty team is that how does Netty perform under a load of large sized incoming payloads. In my performance tests, I am generating the load using a 1000 concurrent devices sending 1MByte sized payloads each. It tends to generate 1GIG of uploaded data in 5 minutes. This load tends to break MINA.

Can Netty support this load and more. Your input on this is greatly appreciated.

btw: I did profile the application using JProfiler. It does point to the MINA stack as the culprit. I even completely disconnected my server side stack and left it at plain MINA server that consumes data and returns data. In this mode, even though there is no real processing going on with my backend, it starts to run out of memory in about 15 to 20 minutes

Upvotes: 12

Views: 1231

Answers (2)

anirudha13
anirudha13

Reputation: 47

I've used Netty in the past for a project, where we were trying to build a platform that would be able to stream large chunks of data over HTTP. We din't get a chance to load test Netty under the scenario that you've outlined. When we were considering which framework Netty was chosen because of its performance and API over MINA and other similar frameworks. There are a bunch of websites that have done Netty vs MINA comparisons. IMO you should be able to satisfy your use case by using a combination of Direct ByteBuffers and writing the data to disk/storage as it is being read.

Upvotes: 0

Colby
Colby

Reputation: 454

It would seem to be as an incorrect usage of the library rather than an actual problem with the library itself. If not user error, IMO Netty would probably do the same thing. I've used MINA in high bandwidth applications before without this result. It may be that your network is saturated, and the buffers just consume all of the memory waiting to upload. Just my 2¢.

Upvotes: 1

Related Questions