CODEWITHSUNDEEP
javaandroidfirebasefirebase-realtime-database
crysis
crysis

Reputation: 1584

Firebase Database: Out of Memory Exception on querying huge amount of data on Android app

I'm using Firebase database for my Android app. I needed a unique 5-digit code which should be valid for 3 days. To resolve this I inserted all possible 5-digit codes in Firebase database where each entry initially has status as 1. So structure looks something like this:

codes: {
  'abcde': {status:1},
  'zhghs': {status:1}
  .....
}

This codes object has close to 5 million entries. I was thinking I would set status to 2 once a code is in use.

I'm using this to get a code from Firebase:

db.getReference("codes")
                .orderByChild("status")
                .limitToFirst(1)
                .addListenerForSingleValueEvent { ..... }

I just need 1 code which has status as 1. The above code is causing OutOfMemoryException so I think it is trying to load all the data on device. Shouldn't it just load one element on device side due to limitToFirst(1) constraint?

P.S. I have tried my hands on Firebase FireStore. It doesn't allow importing huge JSON data as of now without exceeding 20K daily limits.

Edit - I've even set db.setPersistenceEnabled(false) Exact error is :

Firebase Database encountered an OutOfMemoryError. You may need to reduce the amount of data you are syncing to the client. java.lang.OutOfMemoryError: Failed to allocate a 28701016 byte allocation with 16777216 free bytes and 18MB until OOM

Upvotes: 0

Views: 3984

Answers (2)

Alex Mamo
Alex Mamo

Reputation: 138824

I just need 1 code which has status as 1.

In this case change your query to:

db.getReference("codes")
            .orderByChild("status")
            .equalsTo(1)
            .limitToFirst(1)
            .addListenerForSingleValueEvent { ..... }

Even with this constraint you should reduce the amount of data you load into memory because it seems that you are forcing too much data into memory. Alternatively you can set a large heap size by setting android:largeHeap="true" in the Manifest.xml file. Usually this error disappear.

You can also take a look at the discussion in this closely related article here.

As @DougStevenson said, is not really a recommended way to solve OOM errors but as I saw in @FrankvanPuffelen answer, the key to solve the problem is to add ".indexOn": "status" which was initially missing.

Upvotes: 1

Frank van Puffelen
Frank van Puffelen

Reputation: 598827

It sounds like you might not have an index defined on your status property. In that case, the server can't do the ordering/filtering for you, and it will be done in the client. That means the client has to read all data and keep it in memory, explaining the huge memory usage.

The solution is quite simple: define an index on your status property.

{
  "rules": {
    "codes": {
      ".indexOn": "status"
    }
  }
}

Also see the documentation on indexing data.

Upvotes: 3

Related Questions