Reputation: 11
I have to download around 6000 records into the sqlite database from a server. The records are present as JSON
objects. While retrieving them using a BufferedReader
, i get an error as out of memory bound exception. I have searched many sites but could not find an appropriate answer. Please suggest some best solution.
My code is:
URL url = new URL("myurl");
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
//getting error here
line = in.readLine();
System.out.println(line);
JSONArray outerArray = (JSONArray) JSONSerializer.toJSON(line);
System.out.println("size : " + outerArray.size());
I got this Fatal Exception -
E/AndroidRuntime(616): java.lang.OutOfMemoryError: [memory exhausted]
Upvotes: 1
Views: 490
Reputation: 2417
the heap memory may low provided to your app,
you need to register app for larger heap by entry in manifest,
<application android:largeHeap="true"> </application>
http://developer.android.com/guide/topics/manifest/application-element.html#largeHeap
Upvotes: 0
Reputation: 9035
Size of you downloading file is too much to hold in memory
Try doing like this
// download the file
InputStream input = new BufferedInputStream(url.openStream());
OutputStream output = new FileOutputStream();
byte data[] = new byte[1024];
long total = 0;
int count;
while ((count = input.read(data)) != -1) {
total += count;
// publishing the progress....
output.write(data, 0, count);
}
output.flush();
output.close();
//
For more explanation see this
Also
You should use json streaming either with gson or jackson. With Jackson you can use a hybrid approach as well. This would reduce your memory consumption significantly as only the portion of json being parsed is loaded into memory.
https://sites.google.com/site/gson/gson-user-guide
Upvotes: 1
Reputation: 41123
It seems you are trying to download the whole JSON Array in one go and that consume a lot of memory. Rather than doing readLine, try reading the input in a fixed amount chunk of buffer (eg: 1024), if that's enough to make one or more JSON Array, parse it. Then continue reading the next chunk. That way you're not blowing your memory space
Please note this method will only work if you can guarantee your JSON array element won't go past 1024 bytes. Otherwise try adjust this number accordingly / use adjustable data structure
Upvotes: 0