CODEWITHSUNDEEP
javamultithreading
Eric
Eric

Reputation: 399

can multithread works faster?

i have 2 machine (Intel Atom(TM) CPU D525) and each of them running different OS (1 windows 7 and 1 ubuntu 10.04).

I wish to send a stack of image from Windows 7 machine to Ubuntu machine.

I am now using multithread to do it. i have attached my code below:

public class RshScp implements Runnable
{

private StreamHandlers streamHandlers = new StreamHandlers(); 
private String screenFileName;
private int clientIndex;
private SingletonServer ss = null;

public RshScp(String screenFileName, int clientIndex, SingletonServer ss)
{
    this.screenFileName = screenFileName;
    this.clientIndex = clientIndex;
    this.ss = ss;
} 

public void run()
{
    sendFileToClient();
}

public void sendFileToClient()
{
    try 
    {   
        DisplayClient dc = null;
        dc = ss.getClient(clientIndex);          

        String execution = sshFileRSH(dc.getHostName(), dc.getUserName(), screenFileName, dc.getRemoteDirectory(), dc.getLocalDirectory()); 
        log.write(execution);
        Process p1 = Runtime.getRuntime().exec(execution);              
        InputStreamReader isr = new InputStreamReader(p1.getInputStream());
        streamHandlers.checkStreamOutput("From RshScp", isr);
    } catch(Exception e){}
}

//Function to set the RSH SCP command
private String sshFileRSH(String hostName, String userName, String localFileNames, String remoteDirName, String localJobDirectory)
{
    String fileTransferCommand = "scp " + localFileNames;
    //String fileTransferCommand = "rsync --partial --progress --rsh=ssh " + localFileNames[0] + " " + localFileNames[1] + " " + localFileNames[2];

    String destinationCommand = userName + "@" + hostName + ":" + remoteDirName;

    String executionCommand = "";
     executionCommand = fileTransferCommand + " " + destinationCommand;

    return executionCommand;            
} // end function

}//end while class

When I try to send the file to multiple clients, I found that it is slow. It uses 5 seconds to connect and send file to client. And sometimes it still will have image lost.

Is anyone know what actually happen that make the multithread slow? and is there any solution that can make it connect and send faster?

Upvotes: 0

Views: 227

Answers (4)

Martin James
Martin James

Reputation: 24847

'It uses 5 seconds to connect and send file to client'

Ho wlong does it take with one thread?

'And sometimes it still will have image lost' - now that is worrying on any TCP-based transfer. It will not happen on a correctly-implemented system even if you have 50 threads.

On typical systems, transfer rate is limited by the network, (as posted already by many others). A multithreaded transfer can provide the most improvement when copying small files to several machines by means of a protocol that continually connects and disconnects. In such cases, the effects of network latency can be mitigated and a large speedup can be expected.

In the case of transferring large files between just two machines, any improvements generated by avoiding connect/disconnect latency are quite likely to be swamped by increased disk seeks as the hard disk performs overlapped writes to multiple files.

First - fix your transfer! Data should not be lost even with an inappropriate number of threads!

Upvotes: 0

Stephen C
Stephen C

Reputation: 718698

The most likely cause of your problem is that your parallel file transfer commands are causing network congestion problems. It sounds like your problems may be compounded by the "buffer bloat" phenomenon.

What can you do about it?

Well the simplest thing would be reduce the number of parallel network connections until the problem goes way. If necessary, do the transfers one at a time.

If you were really keen, you could use a packet sniffer on your network and see if you could spot a problem. But you'd need to know how to interpret the network traffic.

Anyway, the chances are that the problem is just that you are saturating your network link. The only fix for that is to get a faster connection ... or don't try to send as much at the same time.

Upvotes: 0

Eran Medan
Eran Medan

Reputation: 45715

Is anyone know what actually happen that make the multithread slow? and is there any solution that can make it connect and send faster?

As Makoto said, the first bottleneck will be most likely I/O

Multithreading makes sense where you have "wait" time for I/O that is not "utilized"

e.g. if the program has some work to do while your thread is "blocking" on an I/O action, then adding another thread might make sense

For example, if you read images and write them down to a file, you might get a slight improvement in performance if you use 2 threads in a "consumer producer" job (one reads and puts in a buffer, one reads from the buffer and writes to a file e.g. using a BlockingQueue)

Upvotes: 1

Neil Coffey
Neil Coffey

Reputation: 21795

Well, it's hard to tell exactly because you haven't given us the interesting piece of code that Actually Sends The Bytes Across The Network To The Other Machine. But this is what I'd suggest:

  • find out what your network speed is, and hence how long it will take to send an image to the other machine and receive the result
  • from that, work out if distributing processing among the machines is actually worth it
  • if the calculation suggests that it is worth it, don't overcomplicate how you send the data to the other machine with silly nonsense.

Upvotes: 1

Related Questions