CODEWITHSUNDEEP
javascriptjsonperformancefiltering
Blackard
Blackard

Reputation: 85

JSON filtering for multiple keys runs slow

My code works, but it seems to be running exceptionally slow.

I have an array of values to search and I have a JSON that I'm filtering.

var JSONPull;
var FilterJSON = [];
var TypeIDs = [34,35,36,37,38,39,40,11399,1230,17470,17471,1228,17463,17464,1224,17459,17460,18,17455,17456,1227,17867,17868,20,17452,17453,1226,17448,17449,21,17440,17441,1231,17444,17445,1229,17865,17866,1232,17436,17437,19,17466,17467,1225,17432,17433,1223,17428,17429,22,17425,17426,11396,17869,17870];

fetch('URL API')
    .then(res => res.json())
    .then((out) => {
        JSONPull = out;
        TypeIDs.forEach(function (index){
            FilterJSON = JSONPull.filter((element) => {
                console.log("index: "+index);
                console.log(element);
                console.log("type_id: "+ element.type_id);
                element.type_id === index;
            });
        })
    })

The console.logs are more just to watch the code while testing, but definitely shouldn't be causing these performance issues.

Could you please advise me on performance optimisation?

Upvotes: 0

Views: 106

Answers (2)

Blackard
Blackard

Reputation: 85

I realized I had another issue, where the API limited to 1000 objects per call, so I had to iterate through the type_id to add as a filter the API, my code is below, and works great. Thanks for all of the help everyone.

var mergedJSON = [];
var TypeIDs = [34,35,36,37,38,39,40,11399,1230,17470,17471,1228,17463,17464,1224,17459,17460,18,17455,17456,1227,17867,17868,20,17452,17453,1226,17448,17449,21,17440,17441,1231,17444,17445,1229,17865,17866,1232,17436,17437,19,17466,17467,1225,17432,17433,1223,17428,17429,22,17425,17426,11396,17869,17870];

TypeIDs.forEach(function(index){

fetch(`https://<URL here>&order_type=all&type_id=${index}`)
    .then(res => res.json())
    .then((out) => {
        out.forEach(function (element){
            mergedJSON.push(element);
        })
    })
})

and I apologize for the nonstandard casing, I've always been more of a solitary coder, so as long as my variables haven't been all caps I've sort of accepted it. I'll venture to be more standardized.

Upvotes: 0

Brad
Brad

Reputation: 163478

The way you're doing this now, there's a full loop over every single data item for the number of IDs you're looking for. I'd assume you have far more records than you do IDs that you're looking for, so you should refactor this a bit, and remove a whole callback/closure in the process. Lean on some browser optimized code.

out.filter(element => TypeIDs.includes(element.type_id))

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/includes

Also, just as a point of review... your casing of variables is non-standard and makes it difficult to skim through your code. (Not that the compiler cares...)

And, console.log() can definitely cause performance issues.

Upvotes: 1

Related Questions