CODEWITHSUNDEEP
javaparsingtext
Vincenzo
Vincenzo

Reputation: 168

Improve performance on parsing text file

First of all, sorry for my English and for the length of the message.

I'm writing a simple application in Java for visual cryptography for a school project that takes a schema File and a secret image, then creates n images using the information contained in the schema.

For each pixel in the secret image the application looks for a matrix in the schema file and write m pixels in the n shares (one row for each share).

A schema file contains the matrices (n*m) for every color needed for encoding and it is composed as follows

COLLECTION COLOR 1
START MATRIX 1
RGB
GBR
BGR
END
START MATRIX 2
.....
COLLECTION COLOR 2
START MATRIX 1
XXX
XXX
XXX
END
......
//

This file can be a few lines or many thousands so I can't save the matrices in the application, but I need to always read the file.

To test the performance I created a parser that simply search the matrix looking line by line, but it is very slow.

I thought I'd save the line number of each matrix and then use RandomAccessFile to read it but I wanted to know if there is a more powerful method for doing this.

Thanks

Upvotes: 0

Views: 330

Answers (2)

Kevin Day
Kevin Day

Reputation: 16413

If you are truly dealing with massive, massive input files that exceed your ability to load the entire thing into RAM, then using a persistent key/value store like MapDB may be an easy way to do this. Parse the file once and build of an efficient [Collection+Color]->Matrix map. Store that in a persistent HTree. That'll take care of all of the caching, etc... for you. Make sure to create a good hash function for the Collection+Color tuple, and it should be very performant.

If your data access pattern tends to clump together, it may be faster to store in a B+Tree index - you can play with that and see what works best.

Upvotes: 1

fge
fge

Reputation: 121830

For your schema file, use a FileChannel and call .map() on it. With a little effort, you can calculate the necessary offsets into the mapped representation of the file and use that, or even encapsulate this mapping into a custom structure.

Upvotes: 0

Related Questions