16 million goroutines - "GC assist wait"

Question

I'm running a go program that computes the mandelbrot set. A gouroutine is started for every pixel to compute convergence. The program runs fine for pixelLengthx = 1000, pixelLengthy = 1000. If I run the same code for pixelLengthx = 4000, pixelLengthy = 4000, the program starts printing this after a few dozen seconds:

goroutine 650935 [GC assist wait]:
main.converges(0xa2, 0xb6e, 0xc04200c680)
.../fractals/fractals.go:41 +0x17e
created by main.main
.../fractals/fractals.go:52 +0x2af

The program does not terminate and just continues printing.

package main

import (
    "image"
    "image/color"
    "image/draw"
    "image/png"
    "log"
    "math/cmplx"
    "os"
    "sync"
)

var sidex float64 = 4.0
var sidey float64 = 4.0
var pixelLengthx int = 4000
var pixelLengthy int = 4000
var numSteps int = 100

func converges(wg *sync.WaitGroup, i, j int, m *image.RGBA) {
    wht := color.RGBA{255, 50, 128, 255}

    plx := float64(pixelLengthx)
    ply := float64(pixelLengthy)
    fi := float64(i)
    fj := float64(j)

    c := complex((fi-plx/2)/plx*sidex, (fj-ply/2)/ply*sidey)
    zn := complex(0, 0)
    for k := 0; k < numSteps; k++ {
        zn = cmplx.Pow(zn, 2) + c
    }

    if cmplx.Abs(zn) > 0.1 {
        m.Set(i, j, wht)
    }

    wg.Done()
}

func main() {
    err := Main()
    if err != nil {
        log.Fatal(err)
    }
}

func Main() error {
    m := image.NewRGBA(image.Rect(0, 0, pixelLengthx, pixelLengthy))
    blk := color.RGBA{0, 0, 0, 255}
    draw.Draw(m, m.Bounds(), &image.Uniform{blk}, image.ZP, draw.Src)

    numGoroutines := pixelLengthx * pixelLengthy
    wg := &sync.WaitGroup{}
    wg.Add(numGoroutines)

    for x := 0; x < pixelLengthx; x++ {
        for y := 0; y < pixelLengthy; y++ {
            go converges(wg, x, y, m)
        }
    }

    wg.Wait()

    f, err := os.Create("img.png")
    if err != nil {
        return err
    }
    defer f.Close()

    err = png.Encode(f, m)
    if err != nil {
        return err
    }

    return nil
}

What is going on here? Why does the program even print something?

I'm using go version go1.8 windows/amd64.

Memory usage during program execution.

Answer 1

goroutine is lightweight but you have too much overconfident. You should make worker like below, I think.

package main

import (
    "image"
    "image/color"
    "image/draw"
    "image/png"
    "log"
    "math/cmplx"
    "os"
    "sync"
)

var sidex float64 = 4.0
var sidey float64 = 4.0
var pixelLengthx int = 4000
var pixelLengthy int = 4000
var numSteps int = 100

func main() {
    err := Main()
    if err != nil {
        log.Fatal(err)
    }
}

type req struct {
    x int
    y int
    m *image.RGBA
}

func converges(wg *sync.WaitGroup, q chan *req) {
    defer wg.Done()

    wht := color.RGBA{255, 50, 128, 255}
    plx := float64(pixelLengthx)
    ply := float64(pixelLengthy)

    for r := range q {

        fi := float64(r.x)
        fj := float64(r.x)

        c := complex((fi-plx/2)/plx*sidex, (fj-ply/2)/ply*sidey)
        zn := complex(0, 0)
        for k := 0; k < numSteps; k++ {
            zn = cmplx.Pow(zn, 2) + c
        }

        if cmplx.Abs(zn) > 0.1 {
            r.m.Set(r.x, r.y, wht)
        }
    }
}

const numWorker = 10

func Main() error {
    q := make(chan *req, numWorker)
    var wg sync.WaitGroup
    wg.Add(numWorker)
    for i := 0; i < numWorker; i++ {
        go converges(&wg, q)
    }

    m := image.NewRGBA(image.Rect(0, 0, pixelLengthx, pixelLengthy))
    blk := color.RGBA{0, 0, 0, 255}
    draw.Draw(m, m.Bounds(), &image.Uniform{blk}, image.ZP, draw.Src)

    for x := 0; x < pixelLengthx; x++ {
        for y := 0; y < pixelLengthy; y++ {
            q <- &req{x: x, y: y, m: m}
        }
    }
    close(q)

    wg.Wait()

    f, err := os.Create("img.png")
    if err != nil {
        return err
    }
    defer f.Close()

    err = png.Encode(f, m)
    if err != nil {
        return err
    }

    return nil
}

Answer 2

This is due to the garbage collector attempting a stop-the-world sweep. The 1.8 GC minimizes but doesn't eliminate STW collection. The actual collection is fast, but first it has to preempt all goroutines before it can finish GC. A goroutine can be preempted by the scheduler when it makes a function call. If you're doing all inline math and tight loops, with many goroutines live, this could take a very long time.

Also, as @JimB and @putu noted, while goroutines are extremely resource-efficient and very large numbers have been used in production circumstances, these circumstances have been with extraordinary resources available (e.g. Google's production infrastructure). Goroutines are light weight, but 16M feathers is still going to be heavy. If your system does not have 32GB+ memory, you're likely over-taxing your machine, not Go itself.

Try running with GOTRACEBACK=crash and GODEBUG=gctrace=1 and see if you can get some probative info from the stack trace when it dies.

A web search for "GC assist wait" turned up this useful thread: https://groups.google.com/forum/#!topic/golang-dev/PVwDFD7gDuk