CODEWITHSUNDEEP
gopprof
Nelson
Nelson

Reputation: 181

convert byte[] to string in golang strange occupy heap

I found strange occupy heap when convert byte[] to string with below code

package main

import (
  "bytes"
  "fmt"
  "net/http"
  _ "net/http/pprof"
  "strings"
  "time"
)

var (
  c = make(chan int, 500000)
)

func main() {
  go func() {
    http.ListenAndServe(":8080", nil)
  }()
  f := func(ss []string) {
    fmt.Println(ss)
    time.Sleep(time.Millisecond)
    <-c
  }
  for {
    c <- 1
    bs := bytes.NewBufferString("A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U,V,W,X,Y,Z").Bytes()
    fmt.Println(bs) // will raise memory leak after marked as comment???
    s := string(bs)
    ss := strings.Split(s, ",")
    go f(ss)
  }
}

  1. without fmt.Println(bs) will gradually exhausting memory.

  2. with fmt.Println(bs) work fine. i can't understand what happened? i'm worked with version go1.9.2 darwin/amd64

Upvotes: 2

Views: 1079

Answers (1)

wasmup
wasmup

Reputation: 16223

No, there is no memory leak:
You are using 500000 concurrent goroutines, you just need to limit (reduce) the number of concurrent goroutines, e.g.:

c := make(chan int, runtime.NumCPU())

Try this ( and see the end of this edit):

package main

import (
    "bytes"
    "fmt"
    "runtime"
    "strings"
    "time"
)

func main() {
    c := make(chan int, runtime.NumCPU())
    for {
        c <- 1
        bs := bytes.NewBufferString("A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U,V,W,X,Y,Z").Bytes()
        s := string(bs)
        ss := strings.Split(s, ",")
        go func(ss []string) {
            fmt.Println(ss)
            time.Sleep(time.Millisecond)
            <-c
        }(ss)
    }
}

Your Code:

package main

import (
    "bytes"
    "fmt"
    "net/http"
    _ "net/http/pprof"
    "strings"
    "time"
)

var (
    c = make(chan int, 500000)
)

func main() {
    go func() {
        http.ListenAndServe(":8080", nil)
    }()
    f := func(ss []string) {
        fmt.Println(ss)
        time.Sleep(time.Millisecond)
        <-c
    }
    for {
        c <- 1
        bs := bytes.NewBufferString("A,B,C,D,E,F,G,H,I,J,K,L,M,N,O,P,Q,R,S,T,U,V,W,X,Y,Z").Bytes()
        // fmt.Println(bs) // will raise memory leak after marked as comment???
        s := string(bs)
        ss := strings.Split(s, ",")
        go f(ss)
    }
}

It reaches steady state after a while and even reduced Memory usage:

// Mem          CPU time:
// 5,464,208K 0:1:20
// 5,468,208K 0:2:20
// 5,469,608K 0:3:20
// 5,469,844K 0:4:20
// 5,469,844K 0:5:20
// 5,469,848K 0:6:20
// 5,469,848K 0:7:20 fixed
// 5,469,848K 0:8:20 fixed
// 5,469,616K 0:9:20 reduced

Upvotes: 5

Related Questions