CODEWITHSUNDEEP
javaout-of-memoryfreemarker
GOXR3PLUS
GOXR3PLUS

Reputation: 7255

Stream Huge Lists with FreeMarker , out of memory exception

Description

There is a big database of players 200.000 and below is the .ftl model:

<header> ... <header>
<main>
!Here is the problem
<#list items as vo>
  <name> $vo.name} </name>
  <surname> ${vo.surname} </surname>
  <abilities> ${vo.abilities} </abilites>
</#list>
</main>
<footer>  ... </footer>

So what's wrong ?

The above works good when i have a small amount of players , let's say 5.000.

What i am trying to do ?

I need to create this document at once for all the players which means i have to pass a list with 100.000 or more players . This is going to cause Java OutOfMemoryException.

So i am fetching every time 1.000 players and creating the list , but the problem is .... how to pass all the players as before ? I have one .ftl model and FreeMarker doesn't support something like that ...

I want all the 200.000 players at the same document .

What technique i should use :) ?

The Java class looks like this :

public FinalXMLVo extends HashMap<String,Object>{

 ...
 private List<Player> players;

 public FinalXMLVo(....,List<Player> players){

  ...
  this.put("items",players);

 }

...

}

Upvotes: 0

Views: 530

Answers (1)

ddekany
ddekany

Reputation: 31112

Does the Writer you pass to FreeMarker buffer all the output? Maybe seeing where the OutOfMemoryException usually occurs helps to figure that out.

Other than that, I'm not aware of any reason why a longer list would consume more memory in FreeMarker. At least in such a basic use case you are showing, it should be able to keep render even an infinite list.

Also, if instead of a List you are using an Iterator, that also frees up memory as far as the source of the data (like a ResultSet maybe) is smart enough to not buffer all of it.

Upvotes: 1

Related Questions