CODEWITHSUNDEEP
javaxmlparsingmemory-management
Amila Banuka Amarasinghe
Amila Banuka Amarasinghe

Reputation: 446

managing very large xml in memory

we have an existing system in which there is an HTTP XML api where users can query about objects containing in the system. objects are categorized in hierarchical manner like folders. the system might contain very large number of objects. the API builds an XML model (not direct object to XML mapping) using properties of the objects and returns that via the HTTP API. if the user queries the root of the system, it will return properties of each and every object in the system and maintaining this xml object in memory causes JVM Out of Memory issues frequently.

Currently the system is maintaining jdom based DOM type xml model. the requirement is to construct the object model by iterating the system objects one by one (inefficient but this is a legacy system) and return the complete xml model. is there any memory efficient way of doing this.

As I understand SAX parser is mainly for reading xml objects and not for frequent updates.

Any suggestion would be appreciated

Upvotes: 0

Views: 196

Answers (1)

Kayaman
Kayaman

Reputation: 73548

Creating the model in memory first is inefficient, especially if you then create the XML in memory too. Then you have the data twice in memory and that's a lot of waste.

You can use XMLStreamWriter to avoid creating the full XML in memory, streaming it out directly instead. This should at least allow you to get rid of the DOM model, which is likely to use more memory than the data model.

Upvotes: 1

Related Questions