Swift / SwiftNIO / Vapor: wait() or similar inside EventLoop & equivalent to JavaScript‘s Promise.all()

Question

I‘m starting with Vapor 4 and got stuck at the very beginning of my journey.

I know Promises in JavaScript and I think I have an understanding of Swift‘s Futures. I think my problem is the fact, that sadly most tutorials out there use wait() to keep their examples short and simple. In Vapor I‘m confronted with the EventLoop and wait() being forbidden in there.

What I‘m trying to do

I’m trying to perform some queries on a MySQL database, which need to be executed serially:

1. Two tables are truncated.
2. Then I'm copying all rows from a third table into one of the truncated tables.
3. Finally I'm querying that filled table, try to iterate over each matched result and insert it into the other one of the truncated tables.

What's going wrong / where I need help

1. After some days of hard to understand compile errors it's now running. The first part is being executed but it's missing a correct implementation of some callback. I'm thinking of something like Promise.all([]) in JavaScript. I don't want to nest these two queries because I think it's cleaner to have an array of table names and executing the query for each of them. That's the first minor thing I don't know how to do.
2. Most importantly: the second step, inserting the matched rows into the other table, fails. In the console in Xcode it prints many times:
   [ ERROR ] Connection request timed out. This might indicate a connection deadlock in your application. If you're running long running requests, consider increasing your connection timeout. [database-id: mysql, request-id: F159E838-0E90-4025-929E-596A6A66A502]

I guess there are a couple of better ways to solve this problem, but because I want to learn and thinking of some other tasks I'd like to try to implement I would like to solve it by executing these queries serially.

My code

Controllers/RubricsTreeController.swift

import Fluent
import FluentMySQLDriver
import MySQLNIO
import Vapor

struct RubricsTreeController: RouteCollection {
    func rebuild(req: Request) throws -> EventLoopFuture<[Rubric]> {
        let mysql = req.db as? MySQLDatabase

        // Clear database tables
        let tables = ["rubrics", "rubrics_tree"]
        for table in tables {
            mysql!.simpleQuery("TRUNCATE TABLE `\(table)`")    // <-- HERE …
            // … I´d like to somehow collect each returned Future in an Array …
        }
        // … and wait for all Futures to finish

        // Copy contents from imported `import` into table `rubrics`
        mysql!.simpleQuery("INSERT INTO `rubrics` SELECT * FROM `import`")

        // Iterate over all Rubrics and build the Tree by inserting each as a Node into the Nested Set
        let nestedSet = NestedSet(database: mysql!, table: "rubrics_tree")
        var nestedSetRootId = 1;
        let rubrics = Rubric.query(on: mysql as! Database)
            .filter(\.$level == 0)
            .sort(\.$level)
            .sort(\.$parentId)
            .sort(\.$sorting)
            .sort(\.$id)
            .all()
            .flatMapEachThrowing { rubric -> Rubric in
                try? nestedSet.newRoot(rootId: UInt16(nestedSetRootId), foreignId: UInt64(rubric.id!))
                nestedSetRootId += 1
                return rubric
            }
        return rubrics
    }
}

Helpers/NestedSet.swift

import Fluent
import FluentMySQLDriver
import Vapor

class NestedSet {
    var database: MySQLDatabase
    var table: String

    init(database: MySQLDatabase, table: String) {
        self.database = database
        self.table = table
    }

    func newRoot(id: UUID? = nil, rootId: UInt16, foreignId: UInt64? = nil) throws -> EventLoopFuture<Bool> {
        return database
            .simpleQuery("INSERT INTO `\(table)`(rootId, leftValue, rightValue, nodeLevel, nodeMoved, foreignId) VALUES(\(rootId), 1, 2, 0, 0, \(foreignId ?? 0)")
            .map { _ -> Bool in
                true
            }
    }

//  func newRoot(id: UUID? = nil, foreignId: UInt64? = nil) throws -> EventLoopFuture<EventLoopFuture<Bool>> {
//      return database
//          .simpleQuery("SELECT COALESCE(MAX(rootId), 0) AS highestRootId FROM `\(table)`")
//          .flatMapThrowing { (results: [MySQLRow]) in
//              let highestRootId = (results[0].column("highestRootId")?.uint64)!
//              let rootId = UInt16(highestRootId + 1)
//              return try self.newRoot(id: id, rootId: rootId, foreignId: foreignId)
//          }
//  }
}

I'm curious about your ideas and improvements! :)

Answer 1

My suggestion would be to use the new async/await stuff if you can. This will make the code much easier to write.

However, to do this in EventLoopFuture land, you can use flatten to convert an array of futures to a future array. E.g.

struct RubricsTreeController: RouteCollection {
    func rebuild(req: Request) throws -> EventLoopFuture<[Rubric]> {
        let mysql = req.db as? MySQLDatabase

        // Clear database tables
        let tables = ["rubrics", "rubrics_tree"]
        var truncateResults = [EventLoopFuture<Void>]()
        for table in tables {
            let future = mysql!.simpleQuery("TRUNCATE TABLE `\(table)`").transform(to: ())
            truncateResults.append(future)    
        }
        // … and wait for all Futures to finish
        return truncateResults.flatten(on: req.eventLoop).flatMap {

            // Copy contents from imported `import` into table `rubrics`
            return mysql!.simpleQuery("INSERT INTO `rubrics` SELECT * FROM `import`").flatMap { _ in

                // Iterate over all Rubrics and build the Tree by inserting each as a Node into the Nested Set
                let nestedSet = NestedSet(database: mysql!, table: "rubrics_tree")
                var nestedSetRootId = 1;
                let rubrics = Rubric.query(on: mysql as! Database)
                    .filter(\.$level == 0)
                    .sort(\.$level)
                    .sort(\.$parentId)
                    .sort(\.$sorting)
                    .sort(\.$id)
                    .all()
                    .flatMapEachThrowing { rubric -> Rubric in
                        try? nestedSet.newRoot(rootId: UInt16(nestedSetRootId), foreignId: UInt64(rubric.id!))
                        nestedSetRootId += 1
                        return rubric
                    }
                return rubrics
            }
        }
    }
}

Answer 2

Assuming that mysql!.simpleQuery returns a Future you could collect the Futures into an array as

    let tables = ["rubrics", "rubrics_tree"]
    let futuresArray = tables.map { aTable in 
        mysql!.simpleQuery("TRUNCATE TABLE `\(aTable)`")
    }

To my knowledge there is nothing in Combine like Promise.all() but there is a community supported library called CombineExt that has ZipMany which looks like what you're looking for.