CODEWITHSUNDEEP
javascriptiosperformanceipadindexeddb
Matt
Matt

Reputation: 2863

Improving Performance on massive IndexedDB Insert

We are trying to pre-cache a large sum of data on load of our web application into indexed db. From my performance testing the speed is decent on a desktop browser (e.g. Internet Explorer) where I can insert 10,000 records in around 2 seconds. But comparing the exact same functionality on the iPad it drops to 30 seconds. That comparison just blew my mind.

Does anyone know of any hints or tricks to inserting large data sets into indexedDB. I dont know if it is possible at all but if we could build up a copy of an indexedDB server side with all the data prepopulated and then just shoot it over to the client and it just stores it down to the browser. Is anything along these lines doable?

Thanks

Upvotes: 6

Views: 5777

Answers (3)

Yet Another User
Yet Another User

Reputation: 2885

Some pretty bad IndexedDB performance problems can be caused by a prolonged period of the browser just calling onsuccess callbacks and running into event loop overhead after the work is actually done. The performance pattern observed by my app which was doing this was that it did a bunch of work, then it just went answering thousands of callbacks very inefficiently:

screenshot of chrome profiler showing a bunch of things with readable function names in solid fill, then a bunch of lighter regions almost as if greyed out

The right hand part of this image is the callbacks on every request. The solution to doing that is, of course, to not put a callback on every request, but it was previously unclear to me how to do this.

The way that Dexie.js accomplishes this (for details, see src/dbcore/dbcore-indexeddb.ts) is that it saves the last request (e.g. IDBObjectStore.put, etc) sent and sets an onsuccess callback on that one, which then collects the results from the rest of the requests. Thus, it avoids the callback hell.

Another approach from this is to use the IDBTransaction.oncomplete event, and not worry about the callbacks on the individual requests at all.

(note: yes, I know how old this question is, I had this problem today and wanted to put something more useful for this question which is high in Google results)

Upvotes: 3

Francesco
Francesco

Reputation: 1148

I had problems with massive bulk insert (100.000 - 200.000 records). I've solved all my IndexedDB performance problems using Dexie library. It has this important feature:

Dexie has a kick-ass performance. It's bulk methods take advantage of a not well known feature in indexedDB that makes it possible to store stuff without listening to every onsuccess event. This speeds up the performance to a maximum.

Dexie: https://github.com/dfahlander/Dexie.js

Upvotes: 2

Kristof Degrave
Kristof Degrave

Reputation: 4180

How is your data stored in the indexeddb? Is everything in a single object store of do you use multiple objectstores. Do you need all the cached data immediatly?

If you only have a single object store you can start with storing all the data you initialy need, commit that transaction and start a new for all the rest. This way you can start retrieving the initial data while inserting the rest. IndexedDB is async so it should block you.

If you have multiple object stores you can use the same stratigy. First fill up the objectstore you need immediatly and delay the others.

Or maybe consider using the AppCache API instead of the indexeddb api. Using this you can just cache a javascriptfile containing all the json objects you want to cache. This is more the case when you don't need a lot of querying on the data.

Upvotes: 0

Related Questions