How to persistently store a large file client-side with Javascript in a browser?

Question

How do you store a single large file (~50MB) using Javascript in a browser so that it persists in the user's session across multiple requests?

I have an web application that downloads a JSON-based index file to help perform certain calculations. However, it doesn't get updated very often, so to prevent the user from having to download such a large file every time they load the application, I'd like to store it locally.

I've seen some similar questions asked, and seen IndexedDB recommended for "large data" applications, but this seems more geared towards persistently storing large numbers of small records, not a single large file as I'm trying to do.

I was hoping to use the LocalFileSystem API, which looks to be exactly what I want, but contrary to that Mozilla documentation, it only seems to be supported in Chrome, and even then, all the calls warn you that it's "deprecated".

Answer 1

Perhaps in 2024 onward OPFS (Origin-Private File System) is what you'd want. Though, in March of 2024 the implementation in recent browser is still quite limited. For many devices, the storage limits imposed by the browsers will easily permit 50MB.

Update: I needed to make a demo for some students (well, and because I was too curious for too long!) perhaps it would be helpful to you as well (I didn't really clean up may pasting, pull requests are welcome 😆).

Answer 2

[IndexedDB] seems more geared towards persistently storing large numbers of small records, not a single large file as I'm trying to do.

It's not. That you save 10 thousands small files or one big one doesn't change anything in the fact that IndexedDB is the most suited API to store binary data in a browser (after cache).

If the data you want to store fits in the Storage Limits, then you can fill it all with a single file.

Answer 3

MDN's Client-side storage article discusses the options in detail. Basic sessionStorage should work for you, or localStorage if you want to persist the data across multiple sessions (web page visits).

You'll be storing a large string value (your JSON) and you will need to parse it client side before working with that data. 5MB seems to be the limit considering client compatability.

The keys and the values are always in the UTF-16 DOMString format

Newer Cache API would be best but cross platform compatability support would vary.

Use of Cache and Service Workers is an advanced topic

Assuming plain text, 50MB stores approximately 290 books (100 page book). It seems like you could pare down the 50MB of text (JSON string) that you need to store on the client.