CODEWITHSUNDEEP
javamultithreadingsocketsionio
Sneh
Sneh

Reputation: 3737

Best approach for large file transfer to multiple clients using java sockets

I have researched a lot but could not find any thing proper on the topic and hence asking question here.

I want to build an application like dropbox using java sockets, with a social media website where the files we upload on our shared folder gets automatically downloaded or synced onto all of our friends system which we added in the social media website.

Till now what I have thought is I will have a server running and every time a client connects(logs in) I will start 2 handlers, 1 will be for uploads and 1 will be for download. The DOWNLOAD handler will check for a new files from all my friends(meaning they added new files in their shared directory) every 5 minutes and will sync it. and UPLOAD handler will upload the files on the server sending it as a byte array when the handler receives it from the client. Client sends the data to the server using directory watcher to track changes in the directory.

Now the question is starting 2 threads per client, is it feasible? I think it will slow down the server badly as I will imagine to have like 100 clients let say and it means 200 threads. Can you guys just point me in the right direction on as to what approach I should take, I read about NIO and IO and got confused. Also is there any particular library which can be helpful? I looked at Netty, apache mina but don't understand how they can be helpful.

Thanks in advance :)

Upvotes: 0

Views: 1261

Answers (2)

Ractoc
Ractoc

Reputation: 239

When you think you will have a large number of clients, using the standard Socket and ServerSocket isn't going to work. These require 2 threads per client, as you already pointed out. Eventually, this will eat up all your server resources. What you need is the java.nio package. In there you will find the SocketChannel and the ServerSocketChannel. Through these you can set up non-blocking socket communication. This type of communication is event based. Meaning you can have multiple clients using the same 2 threads on the server for reading and writing.

If you want to know more, check out my Socket programming tutorial. The third part goes into setting up a non-blocking variant and should give you everything you need to get started. If you still have questions, let me know, and I'll see if I can help you out further.

Upvotes: 0

César
César

Reputation: 36

I recommend you to take a look to this article about NIO: https://today.java.net/pub/a/today/2007/02/13/architecture-of-highly-scalable-nio-server.html. Also try to think about scalability, if your server send files... what's the speed of your hard drives? It's more important, I think, than number of threads; but keep an eye over thread lock. Why you want to implement something that web makes so well? If I were you I try to think about a secure proxy better than all that stuff about dealing with bytes. Even If you want to transfer files in multiple parts, you may use multipart zip file and download each part programatically, and then rebuild the file. With this approach you can reuse your infrastructure for web and client; also you can benefit of high IO throughtput of modern web servers.

Upvotes: 1

Related Questions