CODEWITHSUNDEEP
javascriptnode.jsperformanceloops
KdgDev
KdgDev

Reputation: 14549

Nodejs break out of loop with async/bluebird?

So, I've got the following code that works, but I'd like for it to not use a for-loop anymore.

The important bit is the "break" story.

    for (const obj of objects) {
        let found = false;

        for (const item of items) {
            const result = item.get(obj);
            if (typeof result !== 'undefined') {
                found = true;
                break;
            }
        }

        if (!found) {
            notDefined.push(obj);
        }
    }

I've tried it with Bluebird but I'm failing there. Array.prototype.forEach isn't an option, it cannot be stopped.

What else can I try?

EDIT: The point is filling that list of objects, the "notDefined" one. The list represents items not found in a map that has been filled purposefully.

So a bunch of them are tested for a match.

If the item does get found, we should skip to the next group. And of course: stop looping over the items because we've found a match and so there's no point in continuing. No need to keep checking all the remaining items, which may well be several hundred thousands.

EDIT2: The purpose of that "notDefined" list is to write the objects to a text file, that can later be checked manually.

The variable names have been changed away from their originals, as to have something different on here.

The purpose of the code is comparing the contents of several directories on one machine, to the content on several directories on an other machine.

This data comes in as several CSV files. They are all read and a list is generated per CSV file. The data in this CSV file is:

And some other metadata.

However, this list each time does not contain merely the strings of the absolute file path, it contains a Map(), the key being a combination of the 3 mentioned pieces of data, just above.

So we have several CSV files for machine A, and then several CSVs for machine B. And we wish to see if items found on machine A, can also be found on machine B.

That's what I'm trying to do here: loop over all the CSVs and find matching items.

If I find a match, then there's no point to continue checking and I can skip to the next set. That's where the break; comes from.

But I do need to check for every single match and then write a result for all items not found anywhere.

Upvotes: 1

Views: 122

Answers (1)

O. Jones
O. Jones

Reputation: 108806

It looks to me like you have two Arrays, objects and items.

And, it looks to me like you want a decently-performing way to know whether each object is missing from all your Maps. I guess your interest in breaking out of a for-loop is part of your strategy to make this perform decently.

May I suggest another approach? Let's start by making a single Set of all the keys in all your Map objects. Something like this.

const allKeys = new Set()
for (const item of items)
  for (const key of item.keys())
    allKeys.add(key)

Now you run through your array of objects, one by one, and push them into notDefined if they're absent from the allKeys set.

for (const obj of objects)
  if (!allKeys.has(obj)) 
    notDefined.push(obj)

This gets you out of nested iterations on two arrays. And it performs well because Set lookups are fast.

All together, in some perfectly synchronous tight loops (no Promises needed):

const allKeys = new Set()
for (const item of items)
  for (const key of item.keys())
    allKeys.add(key)

for (const obj of objects)
  if (!allKeys.has(obj))
     notDefined.push(obj)

To make it even faster, you could consider building your allKeys set as you read your raw data from your CSV files. That would save the pass over all the elements of items.

Upvotes: 1

Related Questions