CODEWITHSUNDEEP
go
Hard worker
Hard worker

Reputation: 4046

How to concurrently iterate through an sql result set in Golang?

The next() method is sequential, is there a way to concurrently iterate through the loop?

I have a result set of 200k rows that I am looping through sequentially and doing logic on each row and want to split it up.

Upvotes: 9

Views: 10955

Answers (1)

David Budworth
David Budworth

Reputation: 11626

The sql.Rows you get back from your query can't be used concurrently (I believe).

but you can do most of the heavy lifting in goroutines.

Here is an example (non-working, but close), on Play

package main

import "fmt"
import "sql"

type Row struct {
    x string
    y string
    z string
}

func processor(ch chan Row) {
    for row := range <-ch {
        // be awesome
    }
}
func main() {
    ch := make(chan Row)
    // two handler go routines (current processors)
    go processor(ch)
    go processor(ch)
    rows := db.Query("select x,y,z from whatever")
    for rows.Next() {
        var row Row
        if err := rows.Scan(&row.x, &row.y, &row.z); err != nil {
            // do something with error
        } else {
            ch <- row
        }
    }
}

Upvotes: 9

Related Questions