CODEWITHSUNDEEP
javanetwork-programming
Paul Taylor
Paul Taylor

Reputation: 13110

Copying file data over network using Java very slow

My code has to read a portion of data from each file it loads, typically about 100,000 bytes. This works fine for local files but can be really slow over my wifi network, even though my network seems adequate (but not blistering fast)

So I created this simple test:

public void testDataCopySpeed() throws Exception
    {
        File file = new File("Z:\\TestFile.mp3");
        System.out.println("start:"+new Date());
        FileChannel fc = new FileInputStream(file).getChannel();
        ByteBuffer bb = ByteBuffer.allocate(500000);         //1/2 MB
        fc.read(bb);
        System.out.println("end:"+new Date());
    }

Would take less than a second on a local file, but over a minute on a networked file.

So I then tried to test my network speed, I cannot see how to just test the wifi but I tested the internet upload/download speed using http://myspeedtestonline.com/ assuming this would be slower than my actual wifi network. It gave me:

Download Speed:512KB/second Upload Speed :40KB/second

and I ran the same test on another computer and it gave a similar speed

So how is it I can download 1/2 MB of data in one second but it can take a minute to copy 1/2MB of data from one file in Java, the file is hosted on a NAS. ?

EDIT:So I have a couple of good answers below, what I really want to know is what is the best way to get access to the first 100,000 bytes from a set of files for read only access that will work for local and networked files, or should I have different code depending on whether or not it is not networked. Fixing the network is not really the solution, I may be able to fix my network but this software has to work on any computer, many of my customer may not have optiminal networks and would not have the skill to fix their network issues.

Upvotes: 1

Views: 3123

Answers (2)

Christian Schlichtherle
Christian Schlichtherle

Reputation: 3155

You are comparing apples with oranges here. When you access http://myspeedtestonline.com/, the flash plugin is probably using the HTTP protocol, but certainly not CIFS.

When you address a file on a NAS, it's most probably using the CIFS protocol. This protocol is known to have performance problems, especially when implemented on consumer appliances (Buffalo drives, etc.).

Sometimes the MTU size is too big, causing the packets to be fragmented and resent.

So my guess is that Java is not the right address to blame in this case. In any case however, you cannot analyze the problem with a simple Java program. You should use a network sniffer for this.

Upvotes: 0

Peter Lawrey
Peter Lawrey

Reputation: 533530

Can you try a memory mapped file?

File file = new File("Z:/TestFile.mp3");
System.out.println("start:"+new Date());
FileChannel fc = new FileInputStream(file).getChannel();
MappedByteBuffer bb = fc.map(FileChannel.MapMode.READ_ONLY, 0, fc.size());
System.out.println("end:"+new Date());

This might only appear faster, or it may help hide how long it takes.

Upvotes: 1

Related Questions