CODEWITHSUNDEEP
javafileserialization
Dan
Dan

Reputation: 9801

What is the absolute fastest way to read and write strings from a file with Java?

What is the absolute fastest way to read and write strings from a file with Java?

I need to read a file of known format into a String[] — where each line is one item in the array — and then back to the file.

The reading, in particular, must be as fast as possible.

Is there a better way then just using a BufferedReader and reading line by line into an array?

Upvotes: 1

Views: 946

Answers (4)

Goblin Alchemist
Goblin Alchemist

Reputation: 827

Just a crazy idea: you could write the length of each string in the file. Something like:

BufferedInputStream stream=new BufferedInputStream(new FileInputStream("file.bin"));
byte[] buff=new byte[256];
String[] result=new String[10];
for(int i=0;i<10;i++){
    int n=(reader.read()<<8)|reader.read();    // string length (assuming all strings are less than 64K)
    if(buff.length<n) buff=new byte[n];
    reader.read(buff,0,n);
    result[i]=new String(buff,0,n);
}
stream.close();

This will free the BufferedReader from checking every input byte for \n. Though I'm not sure that this will be faster than readLine().

Upvotes: 1

Will Iverson
Will Iverson

Reputation: 2079

Here would be my first pass, assuming that memory is not an issue (ha).

  1. Get the file size as it sits on disk (File.length).
  2. Allocate that size buffer.
  3. Load the whole thing in one shot (InputStream.read(byte[])).
  4. Break that String into substrings entirely in memory.
  5. Do Stuff (tm)
  6. Reverse above to save.

Keep in mind that Java stores character data with UCS-16 internally, which means that your nice ASCII file is going to take x2 the size on disk to account for the "expansion." e.g. You have a 4,124 byte foo.txt file will be at least 8,248 bytes in memory.

Everything else is going to be slower, because the application will be designed to deal with some sort of buffering and wrapping (in particular, to deal with not having enough memory to deal with the file size).

Good luck!

Upvotes: 0

ThomasRS
ThomasRS

Reputation: 8287

Use NIO and UTF-8 encoders/decoders which take advantage of your string statistics and also take advantage of JIT optmizations. I believe aalto out / in are doing this, and I am sure you can find others.

Upvotes: 0

Amir Afghani
Amir Afghani

Reputation: 38571

Consider using Google protobuf.

Upvotes: 3

Related Questions