CODEWITHSUNDEEP
androidout-of-memory
Rick Royd Aban
Rick Royd Aban

Reputation: 904

Android OutOfMemoryException when processing large JSON response

I am getting an OutOfMemoryException when processing a very long JSON response returned by the code below:

HttpClient httpclient = new DefaultHttpClient();
HttpGet httpGETRequest = new HttpGet(url);
HttpResponse response = httpclient.execute(httpGETRequest);
return EntityUtils.toString(response.getEntity());

Even though I use System.gc() before I call the http request, the exception still occurs.

I tried to look for the cause of the problem by saving the response on a notepad and check its size, just to find out that it is a 5mb file.

What are the things that should be done in order to avoid the exception?

Thanks for your help in advance.

Upvotes: 0

Views: 2022

Answers (5)

Rick Royd Aban
Rick Royd Aban

Reputation: 904

In this cause, we should avoid large data responses from server. I requested a pagination functionality from the server and problem solved.

Upvotes: 2

Thilak
Thilak

Reputation: 726

Calling System.GC() is not guaranteed to run GC. In fact its not a good practice to programmatically invoke GC. For parsing large json data, use streaming APIs in whichever library you use to encode/decode json. One such popular library is Jackson processor.

5mb is not that big data to cause out of memory, maybe dump the memory of the app and analyze for memory leaks. You could sump using adb dumpsys Tool. Find more about how to use that in What's the Android ADB shell "dumpsys" tool and what are its benefits?

Upvotes: 1

Haspemulator
Haspemulator

Reputation: 11308

I think if you have problem to load 5Mb string into memory (totally reasonable size), streaming JSON parser will probably not help, because you need to have the result of parsing in memory, which will not necessarily be much smaller. I'd suggest you to do the memory profiling using Android Studio built-in memory monitor, to check how much free memory you have and how you're using it in general.

Upvotes: 0

Archimedes Trajano
Archimedes Trajano

Reputation: 41230

I don't recommend that you use System.gc() on Android. What you would need to do is use streaming and avoid loading the whole content in memory if it can be avoided.

https://google-gson.googlecode.com/svn/trunk/gson/docs/javadocs/com/google/gson/JsonStreamParser.html

If you need to access parts of the JSON data more readily rather than doing it one chunk at at time, you can transform the data as it is being read and store it in a SQL database that you can query later.

http://developer.android.com/training/basics/data-storage/databases.html

However, if you have embedded binary data such as an image as part of the JSON object you can't do anything about it anymore because it will read the embedded binary data into memory as one chunk. There's no easy way around this aside from writing your own JSON streamer.

Generally

This general technique can be applied not just to Android, but even application servers dealing with large input data such as XML as well.

Upvotes: 1

Xjasz
Xjasz

Reputation: 1248

try {
        HttpResponse response = httpclient.execute(httppost);
        HttpEntity entity = response.getEntity();
        inputStream = entity.getContent();
        // json is UTF-8 by default
        BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream, "UTF-8"), 8);
        StringBuilder sb = new StringBuilder();
        String line = null;
        while ((line = reader.readLine()) != null) {
            sb.append(line + "\n");
        }
        result = sb.toString();
        inputStream.close();
    } catch (Exception e) {
        Log.d(TAG, "String was to large" + e.toString());
    }

Upvotes: 0

Related Questions