Mixalis Navridis
Mixalis Navridis

Reputation: 329

Can I get a benefit if upload/download multiple files via different sockets in parallel?

I have 4 nodes. A randomly picked node of them uploads a file with the size of 0.02GB while the rest of them try downloading it. My bandwidth is 100 Mbs and let's assume that all nodes have equivalent bandwidth, also lets assume that average I achieve 30 Mbs download speed and 10 upload speed.

The code works well, and the program is executed sequentially so I need to wait for the random server to upload the file and then wait for the rest to download it. After that the process can start again with the second file. The code works pretty well but it is not enough.

I want to optimize it. So my thought is to use 4 separated threads and lets assume every 1 sec each node uploads a file of size 0.02GB and the rest download, hence at 4 sec to have 4 parallel files that are uploaded and are downloaded in parallel. Will this increase my performance or it is a hole in water because the bandwidth will be splitted equally so the total time will be the same as the sequential process?

Upvotes: 0

Views: 145

Answers (1)

rzwitserloot
rzwitserloot

Reputation: 103608

If the bottleneck in this process is CPU power, then adding threads will help. If it isn't, it'll do nothing, and, likely, the overhead and complexity of smearing the job out of over multiple threads slows things down, often significantly.

In addition, if CPU is the bottleneck in your current code, then likely that is because your code is faulty, for example, it reads / writes single bytes to a chunk-based concept (for example, you call the one-byte .write() method on a FileOutputStream which is incredibly inefficient - wrap that in a BufferedOutputStream), or does a very clunky job on converting bytes to UTF-8, or who knows - you did not include your current code.

Hence, 'is making this code multi-threaded the thing I should be doing to make this code faster'?

99.99% certainty: No.

The bottleneck is likely network or disk. Those usually are the culprits. Multithreading would do absolutely nothing to mitigate those.

Upvotes: 3

Related Questions