CODEWITHSUNDEEP
httpgochunked-encoding
Keeto
Keeto

Reputation: 4198

Reading and streaming responses from http servers that are using chunk transport in go

I have an http server that responds to requests via the chunk protocol. In this simple example, it responds with the time every second. I have another server that forwards requests to this time server and respond with the same chunk protocol.

curl -i localhost:8000 returns the right thing

however curl -i localhost:9000 returns a chunked response but only after three seconds (not every second). This means that somehow io.copy does not really flush the output as soon as it comes

Upvotes: 0

Views: 627

Answers (1)

Mr_Pink
Mr_Pink

Reputation: 109377

io.Copy has to copy the entire contents before you call Flush. If you want the contents sent before the buffer has filled, you need to call flush after each write yourself.

A complete version of io.Copy that flushes an http.ResponseWriter after each write would look like:

func flushCopy(dst io.Writer, src io.Reader) (written int64, err error) {
    buf := make([]byte, 1024 * 8)
    flusher, canFlush := dst.(http.Flusher)
    for {
        nr, er := src.Read(buf)
        if nr > 0 {
            nw, ew := dst.Write(buf[0:nr])
            if nw > 0 {
                if canFlush {
                    flusher.Flush()
                }
                written += int64(nw)
            }
            if ew != nil {
                err = ew
                break
            }
            if nr != nw {
                err = io.ErrShortWrite
                break
            }
        }
        if er == io.EOF {
            break
        }
        if er != nil {
            err = er
            break
        }
    }
    return written, err
}

Upvotes: 2

Related Questions