CODEWITHSUNDEEP
goconcurrencystreamchannel
luqo33
luqo33

Reputation: 8361

Concurrently send bytes coming from stream to two destinations

What's the best way to broadcast values from a stream to two network destination simultaneously? Here is simplified code:

func main() {
    resp, _ := http.Get("http://origin.com/image.jpeg")

    var buf bytes.Buffer
    // tee, when read, writes to &buf
    respBodyTee := io.TeeReader(resp.Body, &buf)

    sendToClient(respBodyTee)
    uploadeToServer(&buf)
}

A stream cannot be read twice, so TeeReader is used to populate &buf with whatever is read from respo.Body.

However, the functions (sendToClient and uploadToServer) will run synchronously while I'd like to them to make their work concurrently.

Solution that I have on mind is to pass a channel to sendToClient that will populate channel with bytes already sent to client. Later have uploadToServer read from the same channel. Something along these lines:

func main() {
    resp, _ := http.Get("http://origin.com/image.jpeg")

    ch := make(chan byte)
    go sendToClient(respBodyTee, ch) // pass 'ch' for writing and run in a goroutine
    uploadeToServer(ch) // will read from 'ch' (synchronous)
}

I'm new to Go and am not sure if the above is the right direction.

Upvotes: 2

Views: 650

Answers (1)

owlwalks
owlwalks

Reputation: 1681

in your scenario, it's better to have 2 independent byte streams for 2 network calls. If they rely on one source stream, when sendToClient stalls uploadeToServer will hang. channel wouldn't solve the problem above but introducing locking overhead.

you can try io.MultiWriter to make 2 independent byte streams

var buf1, buf2 bytes.Buffer
mw := io.MultiWriter(&buf1, &buf2)

if _, err := io.Copy(mw, r.Body); err != nil {
    ...
}
go sendToClient(buf1)
go uploadeToServer(buf2)
...

Upvotes: 1

Related Questions