android dlavikvm-heap out ot memory allocation upon XZ compression

Question

I am trying to compress a tar to a tar.xz on API 2.3.3.

No matter what I do I get this error:

06-09 20:56:32.230: V/CameraDemo(23418): filesPath -- > /mnt/sdcard/myimages
06-09 20:56:32.336: D/dalvikvm(23418): GC_FOR_ALLOC freed 75K, 2% free 9193K/9347K, paused 14ms
06-09 20:56:32.363: I/dalvikvm-heap(23418): Grow heap (frag case) to 21.304MB for 12853537-byte allocation
06-09 20:56:32.390: D/dalvikvm(23418): GC_CONCURRENT freed 1K, 1% free 21744K/21959K, paused 1ms+2ms
06-09 20:56:32.410: D/dalvikvm(23418): GC_FOR_ALLOC freed 1K, 2% free 22005K/22279K, paused 17ms
06-09 20:56:32.476: I/dalvikvm-heap(23418): Grow heap (frag case) to 37.558MB for 16777232-byte allocation
06-09 20:56:32.480: W/dalvikvm(23418): 67108888 byte allocation exceeds the 50331648 byte maximum heap size
06-09 20:56:32.480: I/dalvikvm-heap(23418): Forcing collection of SoftReferences for 67108888-byte allocation
06-09 20:56:32.508: D/dalvikvm(23418): GC_BEFORE_OOM freed 9K, 1% free 38380K/38727K, paused 25ms
06-09 20:56:32.508: E/dalvikvm-heap(23418): Out of memory on a 67108888-byte allocation.
06-09 20:56:32.508: I/dalvikvm(23418): "main" prio=5 tid=1 RUNNABLE
06-09 20:56:32.508: I/dalvikvm(23418):   | group="main" sCount=0 dsCount=0 obj=0x40a53460 self=0xa14828
06-09 20:56:32.508: I/dalvikvm(23418):   | sysTid=23418 nice=0 sched=0/0 cgrp=default handle=1074758792
06-09 20:56:32.508: I/dalvikvm(23418):   | schedstat=( 0 0 0 ) utm=17 stm=8 core=0
06-09 20:56:32.508: I/dalvikvm(23418):   at org.tukaani.xz.lz.BT4.<init>((null):~-1)
06-09 20:56:32.508: I/dalvikvm(23418):   at org.tukaani.xz.lz.LZEncoder.getInstance((null):-1)
06-09 20:56:32.508: I/dalvikvm(23418):   at org.tukaani.xz.lzma.LZMAEncoderNormal.<init>((null):-1)
06-09 20:56:32.511: I/dalvikvm(23418):   at org.tukaani.xz.lzma.LZMAEncoder.getInstance((null):-1)
06-09 20:56:32.511: I/dalvikvm(23418):   at org.tukaani.xz.LZMA2OutputStream.<init>((null):-1)
06-09 20:56:32.511: I/dalvikvm(23418):   at org.tukaani.xz.LZMA2Options.getOutputStream((null):-1)
06-09 20:56:32.511: I/dalvikvm(23418):   at org.tukaani.xz.LZMA2Encoder.getOutputStream((null):-1)
06-09 20:56:32.511: I/dalvikvm(23418):   at org.tukaani.xz.BlockOutputStream.<init>((null):-1)
06-09 20:56:32.511: I/dalvikvm(23418):   at org.tukaani.xz.XZOutputStream.write((null):-1)
06-09 20:56:32.511: I/dalvikvm(23418):   at org.apache.commons.compress.compressors.xz.XZCompressorOutputStream.write(XZCompressorOutputStream.java:71)
06-09 20:56:32.511: I/dalvikvm(23418):   at utilities.CompressTar.compXZ(CompressTar.java:127)
06-09 20:56:32.511: I/dalvikvm(23418):   at utilities.CompressTar.<init>(CompressTar.java:52)
06-09 20:56:32.511: I/dalvikvm(23418):   at com.jadeye.CameraActivity$4.onClick(CameraActivity.java:101)
06-09 20:56:32.519: I/dalvikvm(23418):   at android.view.View.performClick(View.java:3511)
06-09 20:56:32.523: I/dalvikvm(23418):   at android.view.View$PerformClick.run(View.java:14105)
06-09 20:56:32.523: I/dalvikvm(23418):   at android.os.Handler.handleCallback(Handler.java:605)
06-09 20:56:32.523: I/dalvikvm(23418):   at android.os.Handler.dispatchMessage(Handler.java:92)
06-09 20:56:32.523: I/dalvikvm(23418):   at android.os.Looper.loop(Looper.java:137)
06-09 20:56:32.523: I/dalvikvm(23418):   at android.app.ActivityThread.main(ActivityThread.java:4424)
06-09 20:56:32.523: I/dalvikvm(23418):   at java.lang.reflect.Method.invokeNative(Native Method)
06-09 20:56:32.523: I/dalvikvm(23418):   at java.lang.reflect.Method.invoke(Method.java:511)
06-09 20:56:32.523: I/dalvikvm(23418):   at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:784)
06-09 20:56:32.523: I/dalvikvm(23418):   at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:551)
06-09 20:56:32.523: I/dalvikvm(23418):   at dalvik.system.NativeStart.main(Native Method)
06-09 20:56:32.527: D/AndroidRuntime(23418): Shutting down VM
06-09 20:56:32.527: W/dalvikvm(23418): threadid=1: thread exiting with uncaught exception (group=0x40a521f8)
06-09 20:56:32.539: E/AndroidRuntime(23418): FATAL EXCEPTION: main
06-09 20:56:32.539: E/AndroidRuntime(23418): java.lang.OutOfMemoryError
06-09 20:56:32.539: E/AndroidRuntime(23418):    at org.tukaani.xz.lz.BT4.<init>(Unknown Source)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at org.tukaani.xz.lz.LZEncoder.getInstance(Unknown Source)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at org.tukaani.xz.lzma.LZMAEncoderNormal.<init>(Unknown Source)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at org.tukaani.xz.lzma.LZMAEncoder.getInstance(Unknown Source)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at org.tukaani.xz.LZMA2OutputStream.<init>(Unknown Source)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at org.tukaani.xz.LZMA2Options.getOutputStream(Unknown Source)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at org.tukaani.xz.LZMA2Encoder.getOutputStream(Unknown Source)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at org.tukaani.xz.BlockOutputStream.<init>(Unknown Source)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at org.tukaani.xz.XZOutputStream.write(Unknown Source)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at org.apache.commons.compress.compressors.xz.XZCompressorOutputStream.write(XZCompressorOutputStream.java:71)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at utilities.CompressTar.compXZ(CompressTar.java:127)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at utilities.CompressTar.<init>(CompressTar.java:52)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at com.jadeye.CameraActivity$4.onClick(CameraActivity.java:101)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at android.view.View.performClick(View.java:3511)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at android.view.View$PerformClick.run(View.java:14105)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at android.os.Handler.handleCallback(Handler.java:605)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at android.os.Handler.dispatchMessage(Handler.java:92)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at android.os.Looper.loop(Looper.java:137)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at android.app.ActivityThread.main(ActivityThread.java:4424)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at java.lang.reflect.Method.invokeNative(Native Method)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at java.lang.reflect.Method.invoke(Method.java:511)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:784)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:551)
06-09 20:56:32.539: E/AndroidRuntime(23418):    at dalvik.system.NativeStart.main(Native Method)

Here is the code that is causing it:

    public void compXZ (String saveAs, String infile) throws IOException {

    FileInputStream fin = new FileInputStream(infile);
    BufferedInputStream in = new BufferedInputStream(fin);
    FileOutputStream out = new FileOutputStream(saveAs);
    XZCompressorOutputStream xzOut = new XZCompressorOutputStream(out);
    final byte[] buffer = new byte[BUFFER];
    int n = 0;
    while (-1 != (n = in.read(buffer))) {
        xzOut.write(buffer, 0, n);
    }

    xzOut.finish();
    xzOut.close();
    fin.close();
}

I have made several different versions of this but they all yield the same error. I do have an image gallery loading in my application but have tried compressing with out loading the gallery and it yields the same.

Does any1 know what the problem is? what the solution is? OR maybe the code to compress a tar to a tar.xz?

Thank you for you help.

Jadeye.

Answer 1

The XZ compression you're using tries to allocate 67 MB of memory. For most mobile devices, this is too much.

I don't know much about XZ compression and about the XZ utils implementation but it seems that this compression algorithm requires a lot of memory and isn't really suitable for use on today's mobile devices.