CODEWITHSUNDEEP
javaxmlfile
user710818
user710818

Reputation: 24248

Create XML file with large number of nodes (10 million)

I tried to create file for test with 10 000 000 nodes like:

    DocumentBuilderFactory documentBuilderFactory = DocumentBuilderFactory
            .newInstance();
    DocumentBuilder documentBuilder = documentBuilderFactory
            .newDocumentBuilder();
    Document document = documentBuilder.newDocument();
    Element rootElement = document.createElement("root");
    document.appendChild(rootElement);
    for (int i = 1; i <= 10000000; i++) {
        Element em = document.createElement("ch");
        em.appendChild(document.createTextNode("ch_data"));
        rootElement.appendChild(em);
    }
    TransformerFactory transformerFactory = TransformerFactory
            .newInstance();
    Transformer transformer = transformerFactory.newTransformer();
    DOMSource source = new DOMSource(document);
    StreamResult result = new StreamResult(new File("c:/file1.xml"));
    transformer.transform(source, result);

But received error:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at   com.sun.org.apache.xerces.internal.dom.CoreDocumentImpl.
createElement(CoreDocumentImpl.java:620)
    at main.CreatXMLFile.main(CreatXMLFile.java:27)

Does there exist another library for create XML files with more than 10 000 000 nodes in Java?

Upvotes: 2

Views: 5028

Answers (5)

gustafc
gustafc

Reputation: 28865

Use StAX to write the XML as a stream, so that the entire document doesn't need to reside in memory.

Upvotes: 5

Andreas Dolk
Andreas Dolk

Reputation: 114767

For trivial files like that: consider writing the xml file without using any DOM or StAX:

writeToFile("<root>\n");
for (int i = 0; i < 10000000; i++) {
  writeToFile("<ch>" + getData(i) + "</ch>\n");
}
writeToFile("</root>\n");

That's all - you just need a method that writes a String to a file. And a method to get your textual data.

Upvotes: 5

Rupok
Rupok

Reputation: 2082

you can try by increasing memory size for JVM.

There are several ways to create xml file in java.you can find some example in the following link.

http://www.javazoom.net/services/newsletter/xmlgeneration.html

Upvotes: 0

Balanivash
Balanivash

Reputation: 6867

You could try using SAX parser or JDOM

DOM parser creates an internal tree based on the hierarchical structure of the XML data.In SAX's event-based system, the parser doesn't create any internal representation of the document. Instead, the parser calls handler functions when certain events (defined by the SAX specification) take place. These events include the start and end of the document, finding a text node, finding child elements, and hitting a malformed element.

If you need to parse and process huge XML documents, SAX implementations offer more benefits over DOM-based ones.

Upvotes: 1

Petar Ivanov
Petar Ivanov

Reputation: 93030

You might try to increase the memory allocated for the JVM.

But why do you need to have the whole file in memory? If there is not a really good reason for that, you shouldn't do it.

Upvotes: 0

Related Questions