proper code structure of versionchange transaction in onupgradeneeded event

Question

I'm having trouble understanding how much code can be placed inside an onupgradeneeded event and how to ensure that all the individual async changes to the database in that code will always complete before the transaction does.

If I understand the spec correctly, a transaction of mode "versionchange" is automatically created when an onupgradeneeded event is fired in a request to open a database.

Thus, all code written inside the onupgradeneeded event is considered a single transaction; and I assume if it reaches oncomplete it fires the open request's onsuccess event and if it reaches onerror then it fires the open request's onerror event.

I'm confused by how complex that code can be.

For example, is it safe to have another asynchronous event inside the transaction, such as the "objectStore.transaction.oncomplete = function(event) {}" to wait for an object store to be created before attempting to write to it?

Or should the writing of data to an object store created in an onupgradeneeded transaction be performed in the open request's onsuccess event where it is assured to have been already created?

And, as found in an older example in MDN web docs, should there be a db.onerror event inside the onupgradeneeded event any longer? https://developer.mozilla.org/en-US/docs/Web/API/IDBDatabase#setVersion()_.0A.0ADeprecated

Another example is specific to my database. The database has a set of portfolios in which each portfolio has an object store such as port_data_3 and a variable number of module stores such as module_3.2 indicating the second module of the third portfolio. In addition, there is one port_key object store that holds one record per portfolio containing the portfolio name and its unique key, which in this example is 3.

If a user decides to delete portfolio 3 from the database, a version change is required which includes three main steps. 1) The single row for key 3 in the port_key store must be deleted; 2) the port_data_3 store must be deleted; and 3) all stores with a name that begins with 'module_3.' must be deleted.

Something like d = db.objectStoreNames, l = d.length, can be used to loop through d.item(i) to determine which stores should be deleted. But will the transaction always stay open until all these individual deletions have either completed or failed?

Similarly, the addition/deletion of portfolios and the addition/deletion of modules within a portfolio require version changes. Is it safe to have one open function such that the code for all of these version-change types is in one more complex onupgradeneeded event, or would it be better to have separate open functions for each-version change type so that the onupgradeneeded event code is as simple as possible for each?

I guess part of my confusion is due to the fact that a transaction is made up of a number of separate async processes and I don't control how they are grouped into one transaction event. I'd have to use promises or Promise.all or async functions or generators to do something similar in the rest of the code. And it's taxing my little brain to feel confident that I won't miss some type of error that occurs late.

I thought it'd feel safer if the open request was wrapped in a promise and the promise placed inside a try/catch with an await; but that still doesn't change how the onupgradeneeded and its versionchange transaction determine if all those async processes were successful or at least one failed.

Thank you for any direction and explanation you may be able to provide.

Response to answer from Josh

Thank you for your very detailed answer. It helps me a great deal.

It also helps me to understand your answer to one of my questions back in May--perhaps, the very first one I submitted here--that I didn't really understand the latter portion of at that time. The question was How to map mulit-level object to indexedDB for best efficiency.

I think I get it, now, in light of your answer to this question. I can change the database structure such that I never need to upgrade the database due to the user adding portfolios and modules. And, as you recommended in your answer to that earlier question, the three keys needed--portfolio, module, and item under module--could be three indexes and the combination of the three could be an index also, since that will be the query most often needed. In fact, the combination of the three is the only unique identifier that I have, unless I let the browser generate one. Thus, instead of a variable number of multiple module object stores, the addition of each requiring a database upgrade, there would be the equivalent of one module object store that contains the previous data objects across all portfolios and all modules. And the same would be done to combine the multiple portfolio-level stores in one.

That simplifies that database and eliminates the need for the largest part of the onupgradeneeded code/transaction that I was so concerned about. With these changes, I think I can finally get back to updating the program to use indexedDB instead of localStorage and maybe even start sleeping nights again. In this case, the codiing itself is not the challenging part, but the determination of an efficient database structure.

Thanks also for the information on how to better layout future questions.

Answer 1

• You can have a bunch of what you call 'async' stuff in the upgradeneeded event listener. It is safe to listen for the complete event of the versionchange transaction from within the upgradeneeded handler, although I would add this is not worthwhile.
• It is not, however, safe to do something like an awaited fetch call within onupgradeneeded. At runtime the transaction will complete before the call resolves, and all queued up operations will fail.
• A transaction can have multiple requests.
• The transaction will remain open as long as there are pending requests.
• The transaction will generally remain open until the end of the current epoch of the event loop, and sometimes a tiny bit longer after that.
• A transaction completes, automatically, a very short period of time after it detects that there are no pending requests.
• Attempting to queue up an async call like fetch while a transaction is open, and then wait for the fetch to settle, and then do an insert on that open transaction, will not work, because the transaction will complete before that, because that fetch doesn't resolve until at the earliest at the start of the next epoch of the event loop, but the transaction resolves earlier because no new requests were detected.
• Transactions are basically setup to timeout from the start. Each time you make a request, you are keeping them alive a tiny bit longer.
• A transaction resolves/settles/finishes when its request finish. Not just when its request start. A request that has not finished is still pending. A transaction with pending requests will not complete (timeout).
• A request can finish successfully or with an error, both settle the request.
• A transaction with multiple pending requests, where one request fails, may abort the other pending requests and just finish with an error. That one request error becomes the transaction's error. The transaction then finishes with that error.

Note some of careful wording of documentation for functions you find in places like the Mozilla Developer Network (MDN). For example store.delete(thing). The delete function creates a new request in the transaction the store is associated with. This is a perfectly safe async thing to do. You can create any number of additional requests like this. You don't have to wait for the transaction to finish to add a new request. You don't have to wait for the other requests to finish to start a new request. You don't have to wait for a transaction to finish before starting a transaction (with one caveat, the special versionchange transaction in onupgradeneeded).

A transaction is just a grouping of requests. It is a grouping that helps you say that requests live and die together as a group. If any one request fails, the whole group fails. That is the whole point of a transaction. It is so useful, that indexedDB provides you no other way to make requests, you are forced to use a transaction even for just one request. If you have worked with an SQL database, maybe you encountered syntax like START TRANSACTION; SELECT ...; END TRANSACTION;. Same thing here. Except that indexedDB will not allow you do that SELECT ... outside of a transaction, which you were able to do in SQL. Also, indexedDB does not let you end the transaction explicitly yourself. You finish an indexedDB transaction simply by deciding to not create more requests and allowing it to timeout shortly thereafter.

Regarding your overall programming design, I would highly suggest choosing a design where you do not need to change your database schema as your data changes and as things happen in your app/the world. Generally a schema should be constant. If you find yourself needing to create and delete object stores frequently as a normal matter of operation of your app, I would seriously re-think the design.

Also, while you can insert data from within the onupgradeneeded, you should instead generally do this from within the success event handler for the IDBOpenRequest. It is a conventional rule (not a formal one) that changes to data go in onsuccess, and changes to schema go in onupgradeneeded. If you encounter examples on the web where data changes happen in onupgradeneeded, I suggest you read carefully, it is often just done quickly and out of convenience in these examples, not proper app design.

As an aside, you will get better answers if you break up this very broad question into smaller pieces that highlight your areas of confusion, and show example code that works unexpectedly, or with a comment in it highlighting the part you do not know how to formulate.