CODEWITHSUNDEEP
javascriptarraysloopsmemory
Matthew Martini
Matthew Martini

Reputation: 39

Javascript runs out of memory major browsers due to array size

I am trying to create an array, that will be MAASSSIVVEE...i read somewhere that javascript can create an array of up to 4.xx Billion or so. The array i am trying to create will likely be in the quadrillions or higher. I don't even know where to go from here. I am going to assume that JS is not the proper solution for this, but i'd like to give it a try...it is for client side, and i would prefer not to bog down my server with this if there are multiple people using it at once. Also, not looking to learn a new language as i am just getting into JS, and code in general.

could i possibly use setTimeout(),0 breaks in the totalcombos function? Time is not really an issue, i wouldn't mind if it took a few minutes to calculate, but right now it just crashes.

i have tried this using a dedicated worker, but it still crashes the host. the worker code is what i am posting, as the host code is irrelevant to this question (it only compiles the original objects and posts them, then receives the messages back).

The code: (sorry for the mess...im coding noob and just an enthusiast)

onmessage = function(event){
//this has been tested on the very small sample size below, and still runs out of memory
//all the objects in these first arrays are formatted as follows.
// {"pid":"21939","name":"John Smith","position":"QB","salary":"9700","fppg":"23"}
// "PID" is unique to each object, everything else could appear in another object.
// There are no repeated objects.
var qbs = **group of 10 objects like above**
var rbs = **group of 10 objects like above**
var wrs = **group of 10 objects like above**
var tes = **group of 10 objects like above**
var ks = **group of 10 objects like above**
var ds = **group of 10 objects like above**

//This code works great and fast with small sets. ie (qbs, rbs, wrs)
function totalcombos() {
    var r = [], arg = arguments, max = arg.length-1;
    function helper(arr, i) {
        for (var j=0; j<arg[i].length; j++) {
            var a = arr.slice(0); // clone arr
            if(a.indexOf(arg[i][j]) != -1){
                j++;
            } else
                a.push(arg[i][j]);              
            if (i==max) {
                r.push(a);
            } else
                helper(a, i+1);
        }
    }
    helper([], 0);
    return r;
};
//WAY TOO BIG...commented out so as not to crash when run
//var tCom = totalcombos(qbs, rbs, wrs, tes, ks, ds);
//postMessage(tCom.length);
}

When the sets get to be larger like 50 objects in each, it just crashes as it is out of memory. I reduce the set with other code but it will still be very large. How would i fix it? I am trying to create all the possible combinations and then go through and reduce from there based on total salary of each group.

Upvotes: 0

Views: 2226

Answers (1)

Samuel MacLachlan
Samuel MacLachlan

Reputation: 1746

When working with data, regardless of language or platform, its usually best practice to only load the data that's otherwise you encounter errors or bottlenecks etc. as you are finding.

If your data is being stored somewhere like a Database, a JSON file, or a Web Service or an API etc. (anything basically), you'd be better of searching that set of data to retrieve only that which you need, or to at least reduce the size of the Array data your're trying to traverse.

As an analogy, if you're trying to load the whole internet into memory on a PC with only 2GB of RAM, you're going to have a really bad time. :)

Upvotes: 1

Related Questions