CODEWITHSUNDEEP
javascriptperformancevariablesbrowser
user7607751
user7607751

Reputation: 505

What is the best practice to quickly access big amount of data in the web browser?

I have this huge array of strings, saved in a JSON file on a remote server (the file size is 2MB and increasing)..

In my front-end code, I need to constantly loop through this array..

I'm wondering what is the best practice to have the quickest access to it?

My approach right now is as follows:

  1. Fetch the JSON file from the remote server once.
  2. Assign this data to a JavaScript variable.
  3. Save this data to an IndexedDb database, to minimize dealing with the network and save traffic.
  4. When the page reloads, fetch the data from the IndexedDb database and re-assign it to the variable again, then go from there.

So, whenever I need to do the looping in the front-end, I access the variable and loop from there, however, I'm not sure, but, this doesn't sound like a good idea to me. I mean, a variable with 2MB of size?! (it could be 10MB or more at the future), hence, I'm worried if it's using too much memory or it's badly affecting the performance of the web page.

My question is, is there a better way to do this? JavaScript/browser-wise.

And while we're at it, what do I need to know about the best practices for handling big amounts of data in the browser/JavaScript world?

Upvotes: 0

Views: 868

Answers (1)

htho
htho

Reputation: 1799

Don't worry about memory.

Your 10MB of JSON Strings will be transformed to binary representations. Your booleans (true/false) will take 1 bit or maybe 1 byte (depending on the implementation) instead of 4bytes for the strings. Your numbers will take aboit 8 bytes instead of one byte per char. Depending on the implementation, keys will be hashed and/or reused. So the parsed memory footprint will be a lot smaller. Also if the data is static, the js engine will optimize for it. Maybe you can freeze the data to make sure the engine knows the data is immutable.

I wouldn't go as far and store it in a database. You add a lot of overhead. Use the Web Storage API / LocalStorage instead.

Depending on the Contexte the CacheStorage might be interesting (although I have no experience with it).

As for everything you do performance-wise. Don't optimize too early. Make a performance analysis to find the bottlenecks.

Upvotes: 1

Related Questions