CODEWITHSUNDEEP
arraysswiftstruct
Joseph
Joseph

Reputation: 9351

Speeding up creation of nested structs

My code works as intended, I am just trying to speed things up.

I have a database table which contains millions of sensor measurements. When I query the database for the measurements (sorted by the sensor's name) of the past 2 months, I get roughly 300,000 results.

I then proceed to step through the results and want to create nested struct objects from the data. Here some shorter pseudo-code:

struct Measurement {
  var date: Int
  var temp: Double?
  ... 
}

struct Sensor {
  var sensorName: String
  var measurements: [Measurement]
}

var sensors:[Sensor] = []

...

for r in dbResults {
  ...
  let m = Measurement(date: r.date, temp: r.temp1, ... )

  if let index = sensors.firstIndex(where: { $0.sensorName = r.name }) {
    sensors[index].measurements.append(m)
  } else {
    sensors.append(Sensor(sensorName: r.name, measurements: [m]))
  }
}

This is however fairly slow. It takes about a minute for the loop to get done. Is there a way to speed this process up? I am assuming .firstIndex takes too long. I was thinking about creating the objects on parallel threads, but I wasn't sure how to best do that, without risking a race condition when checking the .firstIndex.

Thanks for any ideas

Upvotes: 1

Views: 61

Answers (2)

user652038
user652038

Reputation:

Don't rewrite Dictionary(grouping:by:) yourself. Somebody else maintains that for you for free!!

Put that var here while you're at it: 🗑

let sensors = Dictionary(grouping: dbResults, by: \.name).map {
  Sensor(
    sensorName: $0,
    measurements: $1.map { .init(date: $0.date, temp: $0.temp1) }
  )
}

Upvotes: 0

Asperi
Asperi

Reputation: 257729

I would change storage to dictionary

var sensors:[String: Sensor] = [:]

...

for r in dbResults {
  ...
  let m = Measurement(date: r.date, temp: r.temp1, ... )

  sensors[r.name, default: Sensor(sensorName: r.name, 
      measurements: [])].measurements.append(m)
}

Upvotes: 1

Related Questions