CODEWITHSUNDEEP
javaspringexportods
oscar
oscar

Reputation: 1756

Simple ODS, OutOfMemoryError: Java heap space on save file

I get this error "OutOfMemoryError: Java heap space", when save a file with 1500 rows or more.

I try solve this split the list data

I tried to solve this split data list into smaller parts and save and load the ods file. I use de function insertRowsBefore to append new rows. But its not work for me. When save load the table, append new row list and save, the save function save a full table again and I get the error. My rows has thirty-five columns

I try to use this versions:

<dependency>
<groupId>org.odftoolkit</groupId>
<artifactId>simple-odf</artifactId>
<version>0.6.6</version>
</dependency>

and

<dependency>
<groupId>org.apache.odftoolkit</groupId>
<artifactId>simple-odf</artifactId>
<version>0.7-incubating</version>
</dependency>

Error details:

13:45:12,821 INFO [stdout] Excepción: java.lang.OutOfMemoryError: Java heap space 
13:45:12,822 INFO [stdout] (asyncExecutor-1) at java.util.Arrays.copyOf(Arrays.java:2271) 
13:45:12,822 INFO [stdout] (asyncExecutor-1) at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113)
13:45:12,822 INFO [stdout] (asyncExecutor-1) at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
13:45:12,822 INFO [stdout] (asyncExecutor-1) at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140)
13:45:12,823 INFO [stdout] (asyncExecutor-1) at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
13:45:12,823 INFO [stdout] (asyncExecutor-1) at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282)
13:45:12,823 INFO [stdout] (asyncExecutor-1) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125) 
13:45:12,823 INFO [stdout] (asyncExecutor-1) at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
13:45:12,823 INFO [stdout] (asyncExecutor-1) at java.io.Writer.write(Writer.java:127)
13:45:12,824 INFO [stdout] (asyncExecutor-1) at org.apache.xml.serialize.Printer.printText(Unknown Source)
13:45:12,824 INFO [stdout] (asyncExecutor-1) at org.apache.xml.serialize.XMLSerializer.printAttribute(Unknown Source)
13:45:12,824 INFO [stdout] (asyncExecutor-1) at org.apache.xml.serialize.XMLSerializer.serializeElement(Unknown Source) 
13:45:12,824 INFO [stdout] (asyncExecutor-1) at org.apache.xml.serialize.BaseMarkupSerializer.serializeNode(Unknown Source) 
13:45:12,824 INFO [stdout] (asyncExecutor-1) at org.apache.xml.serialize.XMLSerializer.serializeElement(Unknown Source) 
13:45:12,824 INFO [stdout] (asyncExecutor-1) at org.apache.xml.serialize.BaseMarkupSerializer.serializeNode(Unknown Source) 
13:45:12,825 INFO [stdout] (asyncExecutor-1) at org.apache.xml.serialize.XMLSerializer.serializeElement(Unknown Source) 
13:45:12,825 INFO [stdout] (asyncExecutor-1) at org.apache.xml.serialize.BaseMarkupSerializer.serializeNode(Unknown Source) 
13:45:12,825 INFO [stdout] (asyncExecutor-1) at org.apache.xml.serialize.BaseMarkupSerializer.serializeNode(Unknown Source) 
13:45:12,825 INFO [stdout] (asyncExecutor-1) at org.apache.xml.serialize.BaseMarkupSerializer.serialize(Unknown Source) 
13:45:12,826 INFO [stdout] (asyncExecutor-1) at org.apache.xml.serialize.DOMSerializerImpl.write(Unknown Source)
13:45:12,826 INFO [stdout] (asyncExecutor-1) at org.odftoolkit.odfdom.pkg.OdfPackage.flushDom(OdfPackage.java:1626)
13:45:12,826 INFO [stdout] (asyncExecutor-1) at org.odftoolkit.odfdom.pkg.OdfPackage.getBytes(OdfPackage.java:1566)
13:45:12,826 INFO [stdout] (asyncExecutor-1) at org.odftoolkit.odfdom.pkg.OdfPackage.save(OdfPackage.java:877)
13:45:12,827 INFO [stdout] (asyncExecutor-1) at org.odftoolkit.odfdom.pkg.OdfPackage.save(OdfPackage.java:823)
13:45:12,827 INFO [stdout] (asyncExecutor-1) at org.odftoolkit.simple.Document.save(Document.java:728) 
13:45:12,827 INFO [stdout] (asyncExecutor-1) at org.odftoolkit.odfdom.pkg.OdfPackageDocument.save(OdfPackageDocument.java:228)

Log for my custom functions and services 

13:45:12,828 INFO [stdout] (asyncExecutor-1) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
13:45:12,828 INFO [stdout] (asyncExecutor-1) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
13:45:12,828 INFO [stdout] (asyncExecutor-1) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
13:45:12,828 INFO [stdout] (asyncExecutor-1) | java.lang.OutOfMemoryError: Java heap space 
13:45:12,828 INFO [stdout] (asyncExecutor-1) at java.util.Arrays.copyOf(Arrays.java:2271)
13:45:12,828 INFO [stdout] (asyncExecutor-1) at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:113)
13:45:12,828 INFO [stdout] (asyncExecutor-1) at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
13:45:12,828 INFO [stdout] (asyncExecutor-1) at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:140)
13:45:12,828 INFO [stdout] (asyncExecutor-1) at sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:221)
13:45:12,829 INFO [stdout] (asyncExecutor-1) at sun.nio.cs.StreamEncoder.implWrite(StreamEncoder.java:282)
13:45:12,829 INFO [stdout] (asyncExecutor-1) at sun.nio.cs.StreamEncoder.write(StreamEncoder.java:125)
13:45:12,829 INFO [stdout] (asyncExecutor-1) at java.io.OutputStreamWriter.write(OutputStreamWriter.java:207)
13:45:12,829 INFO [stdout] (asyncExecutor-1) at java.io.Writer.write(Writer.java:127)

My export code:

public static <T> void export(String[] columnTitle, List<T> data, String[] fields, String path, String filename) throws Exception {

    try {
        SpreadsheetDocument ods = SpreadsheetDocument.newSpreadsheetDocument();
        ods.removeSheet(0);

        Table table = Table.newTable(ods, data.size() + MagicNumber.TWO, columnTitle.length, 0, 0);
        table.setTableName(filename);
        String[] values = new String[fields.length];
        int line = 0;

        Border border = new Border(Color.BLACK, BORDER_WIDTH, StyleTypeDefinitions.SupportedLinearMeasure.PT);

        List<Row> rows = table.getRowList();

        Row r = rows.get(line);
        Cell cell = r.getCellByIndex(0);

        //Style for column title
        Font font = new Font(ARIAL_FONT, FontStyle.BOLD, MagicNumber.SEVEN, Color.BLACK);
        r = rows.get(line);
        for (int a = 0; a < fields.length; a++) {
            cell = r.getCellByIndex(a);
            cell.setStringValue(columnTitle[a]);
            cell.setBorders(CellBordersType.ALL_FOUR, border);
            cell.setCellBackgroundColor(Color.valueOf("#A5A5A5"));
            cell.setFont(font);
            cell.setHorizontalAlignment(HorizontalAlignmentType.CENTER);
        }

        //write data
        font = new Font(ARIAL_FONT, FontStyle.REGULAR, MagicNumber.SEVEN);
        for (T obj : data) {
            line++;
            r = rows.get(line);
            for (int i = 0; i < fields.length; i++) {
                cell = r.getCellByIndex(i);
                values[i] = getProperty(obj, fields[i]);
                cell.setStringValue(String.valueOf(values[i]));
                cell.setBorders(CellBordersType.ALL_FOUR, border);
                cell.setFont(font);
                cell.setHorizontalAlignment(HorizontalAlignmentType.RIGHT);
            }
        }

        //Error with 2000 results
        ods.save(path);

    } catch (IllegalAccessException | InvocationTargetException | NoSuchMethodException e) {
        LOG.error(e);
    } catch (Exception e) {
        LOG.error(e);            
    }
}

Upvotes: 0

Views: 650

Answers (1)

slarge
slarge

Reputation: 644

I had to write 200000 rows before getting an GC OutOfMemory error.

To fix this I added a save every 1000 rows (SAVE_POINT below) in the for loop

for (T obj : data) {
    line++;
    r = rows.get(line);

    for (int i = 0; i < fields.length; i++) {
        cell = r.getCellByIndex(i);
        values[i] = getProperty(obj, fields[i]);
        cell.setStringValue(String.valueOf(values[i]));
        cell.setBorders(CellBordersType.ALL_FOUR, border);
        cell.setFont(font);
        cell.setHorizontalAlignment(HorizontalAlignmentType.RIGHT);
    }

    if ((line % SAVE_POINT) == 0) {
        ods.save(path + filename);
    }

}

I have read in a few forums that say simple-odf isn't very good with large spreadsheets so maybe try another library like POI if you have any further problems.

Upvotes: 0

Related Questions