CODEWITHSUNDEEP
javascriptobjectfilterparallel-processinglodash
Doe
Doe

Reputation: 193

How to filter objects in parallel using JavaScript?

Currently, I'm calling a database which returns around 1,000 objects.

I filter the objects before publishing it to the user; and as you can imagine, it takes very long to filter 1,000 objects.

My current filter is like this:

if (!isPatient && this._isMounted) {
  this.setState({
    Users: JSON.parse(JSON.stringify(userSnapData), (k, v) => !v.type || (
      v.type === "Patient" && (_.has(userSnapData[this.state.authUserUID], `Patients`) ? _.has(userSnapData[this.state.authUserUID][`Patients`], k) : true)
    ) ? v : void 0)
  })
}

Where userSnapData is the data that being retrieved from the database.

/* 
 * this is basically saying, filter the object where the type is "Patient"
 * and it has nested "Patients" object
 */
JSON.parse(JSON.stringify(userSnapData), (k, v) => !v.type || (
  v.type === "Patient" && (_.has(userSnapData[this.state.authUserUID], `Patients`) ? _.has(userSnapData[this.state.authUserUID][`Patients`], k) : true)
) ? v : void 0)

The main issue here is how to filter the data in parallel so that whilst the object has been filtered, I want to straight away show that, rather than waiting for the whole 1000 objects to have been filtered.

Is it possible to use some kind of Data.map(async element => {...}) Or something similar?

Update

Suppose I get data like this:

"Users": {
  "w14FKo72BieZwbxwUouTpN7UQm02": {
    "name": "Naseebullah Ahmadi",
    "userType": "Patient",
    "writePermission": false
  },
  "SXMrXfBvexQUXfnVg5WWVwsKjpD2": {
    "name": "Levi Yeager",
    "userType": "Patient",
    "writePermission": false
  },
  "VoxHFgUEIwRFWg7JTKNXSSoFoMV2": {
    "name": "Ernest Kamavuako",
    "userType": "Doctor",
    "writePermission": true
  },
  "hFoWuyxv6Vbt8sEKA87T0720tXV2": {
    "name": "Karla Stanlee",
    "userType": "Doctor",
    "writePermission": true
  }
}

I filter them by userType and return it as an object, and not an array. This is one of the main issues that after I filter them, I require the data back as an object, and an array of objects.

Upvotes: 0

Views: 1581

Answers (1)

Aluan Haddad
Aluan Haddad

Reputation: 31853

Based on your updated question, just write this

const filteredUsers = Object.entries(users)
  .filter(([userId, user]) => user.userType === requiredType)
  .map(([userId, user]) => ({[userId]: user}))
  .reduce(Object.assign, {});

If your filtering is truly expensive, and do you wish to paralelize it you can't do so in JavaScript. Promises won't help you at all as they're about interleaved IO, not parallelyzing collection operations. They don't do anything for that except make it slower if anything.

Free compute intensive collection operations, use of language and run time that support multithreaded programming.

Upvotes: 1

Related Questions