CODEWITHSUNDEEP
javastringparsingio
P basak
P basak

Reputation: 5004

reading large files using apache common io

Hi I am using following code to get all the lines in a list of strings using the FileUtils.readLines functionality from Apache commons IO library. here is my code,

List<String> lines=FileUtils.readLines(new File(fileName));

But whenever i send a file say 45MB with 1 million lines it gives me an out of memory error. What should be the solution. I need to process each individual line.

Upvotes: 2

Views: 4921

Answers (2)

Adolfo Ruiz Ruiz
Adolfo Ruiz Ruiz

Reputation: 13

Its because you are running an enviroment 32 bits instead of 64 bits an also not using the VM arguments as follows: 

-d64 -Xmx6G

 IOUtils.toString((InputStream) new FileInputStream("C://exocerebro//userrecord.sql"), "UTF-8");

Upvotes: 0

Ren&#233; Link
Ren&#233; Link

Reputation: 51353

Read one line after the other

LineIterator it = FileUtils.lineIterator(file, "UTF-8");
 try {
   while (it.hasNext()) {
    String line = it.nextLine();
     // do something with line
   }
 } finally {
  it.close();
 }

Upvotes: 10

Related Questions