Golang Channel Streams

Question

I am relatively new to Golang and don't fully understand streams. I have a function (circuit breaker function) that is making Rest calls. I have it working but it is only streaming the "responseBody" back. I would actually like to stream back the entire request of stream back both the Body and the Header together.

When I try to use a similar approach on the "header" then I get an error that the header is not streamable.

Is there a good/best way to accomplish this? Thank you. Below is my function.

func CallWithRetries(req *http.Request, output chan []byte) error {
    r := retrier.New(retrier.ConstantBackoff(RETRIES, 100 * time.Millisecond), nil)
    attempt := 0
    err := r.Run(func() error {
        attempt++
        resp, err := Client.Do(req)
        if err == nil && resp.StatusCode < 299 {
            responseBody, err := ioutil.ReadAll(resp.Body)
            if err == nil {
                output <- responseBody
                return err
            }
            return err
        } else if err == nil {
            customLogger.LogDebug("Status code was: " , transactionId)
            err = fmt.Errorf("Status was %v", resp.StatusCode)
        }
        return err
    })
    return err
}

Answer 1

You are looking for the httputil.DumpResponse function.

The code might be changed to something similar to

func CallWithRetries(req *http.Request, output chan []byte) error {
    r := retrier.New(retrier.ConstantBackoff(RETRIES, 100*time.Millisecond), nil)
    attempt := 0
    err := r.Run(func() error {
        attempt++
        resp, err := Client.Do(req)
        if err == nil && resp.StatusCode < 299 {
            dump, err := httputil.DumpResponse(resp, true)
            if err == nil {
                output <- dump
                return err
            }
            return err
        } else if err == nil {
            customLogger.LogDebug("Status code was: ", transactionId)
            err = fmt.Errorf("Status was %v", resp.StatusCode)
        }
        return err
    })
    return err
}

Side notes,

• you might want to consider to close the response body as mentioned in the documentation https://golang.org/pkg/net/http/#Client.Get

• Looks likes the err variable is shadowed, this should be modified to avoid any surprises.

This version of the code attempts to return errors early, and to close the response body. It was not tested, only written on the fly, to use with care.

func CallWithRetries(req *http.Request, output chan []byte) error {
    r := retrier.New(retrier.ConstantBackoff(RETRIES, 100*time.Millisecond), nil)
    attempt := 0
    return r.Run(func() error {
        attempt++
        var resp *http.Response
        {
            r, err := Client.Do(req)
            if err != nil {
                return err
            }
            defer r.Body.Close()
            if resp.StatusCode > 299 {
                customLogger.LogDebug("Status code was: ", transactionId)
                return fmt.Errorf("Status was %v", resp.StatusCode)
            }
            resp = r
        }
        var out []byte
        {
            x, err := httputil.DumpResponse(resp, true)
            if err != nil {
                return err
            }
            out = x
        }
        output <- out
        return nil
    })
}