GZIP Decompressing String Magic Number Exception

Question

I'm trying to decompress a String but always get the magic number exception while decompressing. First I'm compressing a String then it gets base64encoded and later decodes and decompressed. The code is in call order.

It is an android based project and has no external dependencies.

The Encoding and Decoding works fine.

Does someone notice a error in this code and tell me how to resolve it?:

//Compressing a String
public String compress(String s) {
  if(s == null || s.length() == 0) { return string; }

  try {
    ByteArrayOutputStream out = new ByteArrayOutputStream();
    GZIPOutputStream gzip = new GZIPOutputStream(out);
    gzip.write(s.getBytes("UTF-8"));
    gzip.close();
    return out.toString("UTF-8");
  } catch (Exception e) {
    e.printStackTrace();
  }
}
//Encode to Base64
public String encodeBase64(String s) {
  return Base64.encodeToString(s.getBytes("UTF-8"), Base64.NO_WRAP);
}

//Decode Base64
public String decodeBase64(String s) {
  return new String(Base64.decode(s, Base64.NO_WRAP), "UTF-8");
}

public String decompress(String s) {
  if(s == null || s.length() == 0) {
    return s;
  }
  byte[] ba = s.getBytes("UTF-8");
  byte[] buffer = new byte[1024];

  try {
    ByteArrayOutputStream out = new ByteArrayOutputStream(ba.length);
    ByteArrayInputStream in = new ByteArrayInputStream(ba);
    GZIPInputStream gzip = new GZIPInputStream(in); // Magic Number Exception occures here
    int len;
    while((len = gzip.read(buffer)) > 0) {
      out.write(buffer,0 ,len);
    }
    gzip.close();
    out.close();
    return out.toString("UTF-8");
  } catch (Exception e) {
    e.printStackTrace;
  } 
}

UPDATE:

I made a Testclass with the methods and a callmethod that should be usable in android projects:

import android.util.Base64;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.UnsupportedEncodingException;
import java.util.zip.GZIPInputStream;
import java.util.zip.GZIPOutputStream;

public class CompressTest {

    public static final String message = "Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.   
" +
            "
" +
            "Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse molestie consequat, vel illum dolore eu feugiat nulla facilisis at vero eros et accumsan et iusto odio dignissim qui blandit praesent luptatum zzril delenit augue duis dolore te feugait nulla facilisi. Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat volutpat.   
" +
            "
" +
            "Ut wisi enim ad minim veniam, quis nostrud exerci tation ullamcorper suscipit lobortis nisl ut aliquip ex ea commodo consequat. Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse molestie consequat, vel illum dolore eu feugiat nulla facilisis at vero eros et accumsan et iusto odio dignissim qui blandit praesent luptatum zzril delenit augue duis dolore te feugait nulla facilisi.   
" +
            "
" +
            "Nam liber tempor cum soluta nobis eleifend option congue nihil imperdiet doming id quod mazim placerat facer possim assum. Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat volutpat. Ut wisi enim ad minim veniam, quis nostrud exerci tation ullamcorper suscipit lobortis nisl ut aliquip ex ea commodo consequat.   
" +
            "
" +
            "Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse molestie consequat, vel illum dolore eu feugiat nulla facilisis.   
" +
            "
" +
            "At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, At accusam aliquyam diam diam dolore dolores duo eirmod eos erat, et nonumy sed tempor et et invidunt justo labore Stet clita ea et gubergren, kasd magna no rebum. sanctus sea sed takimata ut vero voluptua. est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat.   
" +
            "
" +
            "Consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus.   
" +
            "
" +
            "Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.   
" +
            "
" +
            "Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse molestie consequat, vel illum dolore eu feugiat nulla facilisis at vero eros et accumsan et iusto odio dignissim qui blandit praesent luptatum zzril delenit augue duis dolore te feugait nulla facilisi. Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat volutpat.   
" +
            "
" +
            "Ut wisi enim ad minim veniam, quis nostrud exerci tation ullamcorper suscipit lobortis nisl ut aliquip ex ea commodo consequat. Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse molestie consequat, vel illum dolore eu feugiat nulla facilisis at vero eros et accumsan et iusto odio dignissim qui blandit praesent luptatum zzril delenit augue duis dolore te feugait nulla facilisi.   
" +
            "
" +
            "Nam liber tempor cum soluta nobis eleifend option congue nihil imperdiet doming id quod mazim placerat facer possim assum. Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat volutpat. Ut wisi enim ad minim veniam, quis nostrud exerci tation ullamcorper suscipit lobortis nisl ut aliquip ex ea commodo";

    //Compressing a String
    public static String compress(String s) {
        if(s == null || s.length() == 0) { return s; }

        try {
            ByteArrayOutputStream out = new ByteArrayOutputStream();
            GZIPOutputStream gzip = new GZIPOutputStream(out);
            gzip.write(s.getBytes("UTF-8"));
            gzip.close();
            return out.toString("UTF-8");
        } catch (Exception e) {
            e.printStackTrace();
            return null;
        }
    }
    //Encode to Base64
    public static String encodeBase64(String s) throws UnsupportedEncodingException {
        return Base64.encodeToString(s.getBytes("UTF-8"), Base64.NO_WRAP);
    }

    //Decode Base64
    public static String decodeBase64(String s) throws UnsupportedEncodingException {
        return new String(Base64.decode(s, Base64.NO_WRAP), "UTF-8");
    }

    public static String decompress(String s) throws UnsupportedEncodingException {
        if(s == null || s.length() == 0) {
            return s;
        }
        byte[] ba = s.getBytes("UTF-8");
        byte[] buffer = new byte[1024];

        try {
            ByteArrayOutputStream out = new ByteArrayOutputStream(ba.length);
            ByteArrayInputStream in = new ByteArrayInputStream(ba);
            GZIPInputStream gzip = new GZIPInputStream(in); // Magic Number Exception occures here
            int len;
            while((len = gzip.read(buffer)) > 0) {
                out.write(buffer,0 ,len);
            }
            gzip.close();
            out.close();
            return out.toString("UTF-8");
        } catch (Exception e) {
            e.printStackTrace();
            return null;
        }
    }
}

And here the call:

try {
            String s = CompressTest.message;
            System.out.println("-----------------Message is:-----------------");
            s = CompressTest.compress(s);
            System.out.println("-----------------Compression is:-----------------");
            System.out.println(s);

            s = CompressTest.encodeBase64(s);
            System.out.println("-----------------Base64 Encode is:-----------------");
            System.out.println(s);

            s = CompressTest.decodeBase64(s);
            System.out.println("-----------------Base64 Decode is:-----------------");
            System.out.println(s);

            s = CompressTest.decompress(s);
            System.out.println("-----------------decompressed is:-----------------");
            System.out.println(s);
        } catch (UnsupportedEncodingException e) {
            e.printStackTrace();
        } catch (Exception e) {
            e.printStackTrace();
        }

Furthermore i tested with the data in above testclass.

Message: see above code Compressed: ��Mn�0��9��vtS��cq�L��8MN�7�\�A[��"��-Y$�o�h�TI�c�D��R�i'N�74�ܤK7��c��n�I��(�l�Dk*��h>j��:EޡI�%>d&��h�X@��%�؍�t��(��y��N_��B�ʲ��\��,m�3D��L�l'�P%o��:?hbG[t9Se1��6��-��)#ޒO�TT��b�y��a��{P�2N��m��u5�%A�$ϐ�EN�@�y�r��o�d�[��"��mlVrM��G�9ݞ��p<�?f"'�h9��-��o?c��,�/��/�O<��L5Ntl�9'�$O��a �5f��>8�/� c ��i=v_Q��4��2��Q��

Encoded:

H++/vQgAAAAAAAAA77+9TW7vv70wEO+/ve+/vTnvv70c77+977+9He+/vXYZdFPvv70AY3Hvv71M77+9H++/ve+/vThNTu+/vTfvv71c77+9QVvvv73vv70iDu+/ve+/vS1ZJO+/vW/vv71o77+9VEnvv71j77+9RO+/ve+/vVLvv71pJ07vv703NO+/vdykSzfvv73vv70BYwbvv70H77+977+977+9bu+/vUnvv73vv73vv70o77+9bO+/vURrKu+/ve+/ve+/vRHvv71oPmrvv73vv73vv706Rd6hCEnvv70LCCU+ZCbvv73vv71o77+9WEAq77+977+9Je+/vdiN77+9dO+/ve+/vSjvv73vv73vv73vv715GO+/ve+/ve+/vU5f77+977+9Qu+/vcqy77+977+9XO+/ve+/ve+/vSxt77+9M0Tvv70A77+9TA/vv70CHWwn77+9UCVv77+9Fu+/ve+/vTo/aGI8b++/ve+/vW7vv73vv70677+977+977+977+9D++/vVZOK++/vX/vv71EdHPvv73RtBEbLEJXJO+/vTDvv71qcDTvv73vv71M77+9Q++/ve+/vWnvv705Whzvv71zFx/vv73vv73vv70177+9LxHvv71UZu+/ve+/ve+/ve+/vX0177+977+9VBbvv73vv73vv71iB++/vU7vv71iZO+/ve+/vVHvv73vv70/Qe+/vWcqCS7vv71aJywlKO+/ve+/vSFra++/ve+/vdGUdu+/vXPvv73vv73vv73vv700QVscMXcUfHnvv70aKUjvv73vv71e77+9YHDvv70+FyUQ77+9SlhfK++/ve+/vQzvv70d77+977+977+977+9c3heZwfvv73Juu+/ve+/vWltU++/vTQPF++/vUHvv71fRmdJzpTvv70+77+90qMv77+977+977+977+9YCfvv70wJe+/vSNuOW3vv71FQ++/vdarBe+/vW9SB0UU77+977+9TG4uDe+/ve+/vRDvv70Makfvv73vv70F77+977+9PkdbdDlTZR0x77+977+9Nu+/vQTvv73vv73vv73vv70t77+9Ge+/vSkj3pJP77+9VFTvv73vv73vv73vv70BYu+/vXnvv70Y77+977+9YR3vv73vv717UO+/vTJO77+977+9be+/ve+/ve+/vR51Ne+/vSVB77+9DCTPkAbvv71FThLvv71A77+9GHnvv71y77+977+9bxrvv71k77+9W++/ve+/ve+/vSLvv73vv73vv71tbFZyTRvvv73vv71H77+9Od2e77+977+9cDzvv70/ZiInF++/vWgBOe+/ve+/ve+/ve+/ve+/ve+/ve+/vRkt77+977+9bz9j77+977+9LO+/vS/vv73vv70q77+9L++/vU8877+9BgDvv73vv73vv70E77+977+9TDVOdGzvv705J++/vSRP77+977+9YQ3vv701Zu+/ve+/vT4477+9L++/vQpjDe+/ve+/vWk9dl9R77+977+977+9NO+/ve+/ve+/vTLvv70e77+977+977+977+977+9He+/ve+/vVEY77+9FwAA

Decoded:

��Mn�0��9��vtS��cq�L��8MN�7�\�A[��"��-Y$�o�h�TI�c�D��R�i'N�74�ܤK7��c��n�I��(�l�Dk*��h>j��:EޡI�%>d&��h�X@��%�؍�t��(��y��N_��B�ʲ��\��,m�3D��L�l'�P%o��:?hbG[t9Se1��6��-��)#ޒO�TT��b�y��a��{P�2N��m��u5�%A�$ϐ�EN�@�y�r��o�d�[��"��mlVrM��G�9ݞ��p<�?f"'�h9��-��o?c��,�/��/�O<��L5Ntl�9'�$O��a �5f��>8�/� c ��i=v_Q��4��2��Q��

Here its not really visible but i compared compression before encode with after decode and it looks like this:

one character at the start is missing everything else is fine. So thats probably the part of the data header wich contains the information of gzip format. I'll try to add that manually later and post the result

Solution

For everyone that is interested how i changed the code here the bytearray version of the methods as suggested by Teemu Ilmonen:

//Compressing a String
    public static byte[] compress(String s) {
        if(s == null || s.length() == 0) { return null; }

        try {
            ByteArrayOutputStream out = new ByteArrayOutputStream();
            GZIPOutputStream gzip = new GZIPOutputStream(out);
            gzip.write(s.getBytes("UTF-8"));
            gzip.close();
            return out.toByteArray();
        } catch (Exception e) {
            e.printStackTrace();
            return null;
        }
    }
    //Encode to Base64
    public static String encodeBase64(byte[]  array) throws UnsupportedEncodingException {
        return Base64.encodeToString(array, Base64.NO_WRAP);
    }

    //Decode Base64
    public static byte[] decodeBase64(String s) throws UnsupportedEncodingException {
        return Base64.decode(s, Base64.NO_WRAP);
    }

    public static String decompress(byte[] array) throws UnsupportedEncodingException {
        if(array == null || array.length == 0) {
            return null;
        }
        byte[] buffer = new byte[1024];

        try {
            ByteArrayOutputStream out = new ByteArrayOutputStream(array.length);
            ByteArrayInputStream in = new ByteArrayInputStream(array);
            GZIPInputStream gzip = new GZIPInputStream(in); // Magic Number Exception occures here
            int len;
            while((len = gzip.read(buffer)) > 0) {
                out.write(buffer,0 ,len);
            }
            gzip.close();
            out.close();
            return out.toString("UTF-8");
        } catch (Exception e) {
            e.printStackTrace();
            return null;
        }
    }

Teemu Ilmonen · Accepted Answer

Once you have compressed something with gzip you should not convert it string anymore. You are probably garbling the content there. You should work with byte[] instead:

return out.toByteArray();

GZIP Decompressing String Magic Number Exception

Answers (2)

Related Questions