CODEWITHSUNDEEP
memory-leaksgochannel
hey
hey

Reputation: 7829

channels and memory leaks

I'm trying to develop a program which runs continuously.

It should pull some data from a database every sleepPool seconds and 'process' the information in a non-blocking way(at least that's what I'm trying to do).

The issue is that the memory keeps growing so I'm wondering if I'm doing something wrong.

Below is a snippet from the my program.

var uCh = make(chan *user, buffLimit)     //emits new users to process
var statsCh = make(chan *user, buffLimit) //emits new users to store

func main() {

    go emitUser(db)
    go consumeUser(db)

    for ur := range statsCh {
      log.Infoln(ur)
    }
}

func emitUser(db *sql.DB) {
    for {
        time.Sleep(sleepPool * time.Second)
        log.Infoln("looking for new users")
        rows, err := rowsD.Query()
        for rows.Next() {
            uCh <- usr
        }
    }
}

func consumeUser(db *sql.DB) {
    for usr := range uCh {
        go func(usr *user) {
            //do something with the user
            statsCh <- usr
        }(usr)
    }
}

I've read that I may need to close the channels so that the gc can recycle the memory but I'm not sure how to do that (because the program should run continuously) and if I really need to do it because the data is always read (guaranteed by the range from main) so I assume the memory is recycled.

Upvotes: 0

Views: 646

Answers (2)

OneOfOne
OneOfOne

Reputation: 99254

You didn't give enough time for the GC to kick in, wait for an hour then check the memory.

If you really really wan (bad idea and gonna slow your program) to force it to free the memory you can use something like:

import "runtime/debug"
//........
func forceFree() {
    for _ = range time.Tick(30 * time.Second) {
        debug.FreeOSMemory()
    }
}

func init() {
    go forceFree()
}

Upvotes: 3

ams
ams

Reputation: 25579

This code does a lot of harmless passing things around, but should not leak anything.

My guess is that the rowsD.Query method has some kind of cache, or other leak.

Of course, it could just be fragmentation, or an artefact of the garbage collector (in which case you should see memory use level off, or even drop, over time).

Upvotes: 0

Related Questions