CODEWITHSUNDEEP
javaperformancesortingcollectionsscalability
Ruper
Ruper

Reputation: 135

Efficient and scalable way to sort large amount of strings in Java

I am looking for some ideas idea on sorting large amount of strings from an input file and print out the sorted results to a new file in Java. The requirement is that the input file could be extremely large. I need to consider the performance in the solution, so any ideas?

Upvotes: 0

Views: 2610

Answers (3)

Peter Lawrey
Peter Lawrey

Reputation: 533482

Break the file into amounts you can read in memory. Sort each amount and write to a file. (If you could fit everything into memory you are done) Merge sort the resulting files into a single sorted file.

You can also do a form of radix sort to improve CPU efficiency, but the main bottleneck is all the re-writing and re-reading you have to do.

Upvotes: 1

700 Software
700 Software

Reputation: 87763

Is an SQL database available? If you inserted all the data into a table, with the sortable column or section indexed, you may (or may not) be able to output the sorted result more efficiently. This solution may also be helpful if the amount of data, outweighs the amount of RAM available.

It would be interesting to know how large, and what the purpose is.

Upvotes: 1

Aravind Yarram
Aravind Yarram

Reputation: 80176

External Sorting technique is generally used to sort huge amounts of data. May be this is what you need.

externalsortinginjava is the java library for this.

Upvotes: 3

Related Questions