Reputation: 8520
I can't figure out if I'm doing something wrong or if I'm just pushing it to hard.
I'm trying to sync ~70000 records from my online db to IndexedDB in combination with EventSource and a Worker.
So I get 2000 records per package and then use the following code to store them in IndexedDB:
eventSource.addEventListener('package', function(e) {
var data = JSON.parse(e.data);
putData(data.type, data.records);
});
function putData(storeName, data) {
var store = db.transaction([storeName], 'readwrite').objectStore(storeName);
return new Promise(function(resolve, reject) {
putRecord(data, store, 0);
store.transaction.oncomplete = resolve;
store.transaction.onerror = reject;
});
}
function putRecord(data, store, recordIndex) {
if(recordIndex < data.length){
var req = store.put(data[recordIndex]);
req.onsuccess = function(e) {
recordIndex += 1;
putRecord(data, store, recordIndex);
};
req.onerror = function() {
self.postMessage(this.error.name);
recordIndex += 1;
putRecord(data, store, recordIndex);
};
}
}
It all works for about ~10000 records. Didn't really test where the limit is though. I suspect that at some point there are too many transactions in parallel which causes a single transaction to be very slow and thus causing trouble because of some timeout. According to the dev tools the 70000 records are around 20MB.
Complete error:
Uncaught TransactionInactiveError: Failed to execute 'put' on 'IDBObjectStore': The transaction has finished.
Any ideas?
Upvotes: 1
Views: 169
Reputation: 8357
I don't see an obvious error in your code, but you can make it much simpler and faster. There's no need to wait for the success of a previous put()
to issue a second put()
request.
function putData(storeName, data) {
var store = db.transaction([storeName], 'readwrite').objectStore(storeName);
return new Promise(function(resolve, reject) {
for (var i = 0; i < data.length; ++i) {
var req = store.put(data[i]);
req.onerror = function(e) {
self.postMessage(e.target.error.name);
};
}
store.transaction.oncomplete = resolve;
store.transaction.onerror = reject;
});
}
It is possible that the error you are seeing is because the browser has implemented an arbitrary time limit on the transaction. But again, your code looks correct, including the use of Promises (which are tricky with IDB, but so far as I can tell you're doing it correctly!)
If this is still occurring I second the comment to file a bug against the browser(s) with a stand-alone repro. (If it's happening in Chrome I'd be happy to take a look.)
Upvotes: 1
Reputation: 4180
I think this is due the implementation. If you read the specs a transaction must keep a list of all the requests made in the transaction. When the transaction is commited all these changes are persisted otherwise the transaction will be aborted. Specs
Probably is the maximum request list in your case a 1000 request. You can easily test that by trying to insert a 1001 records. So my guess is when the 1000 request is reached, the transaction is set to inactive.
Maybe change your stratigy and only make 1000 request in every transaction and start a new transaction when the other one is completed.
Upvotes: 0