CODEWITHSUNDEEP
gotimeouthttp-status-codes
Integralist
Integralist

Reputation: 6165

How to handle HTTP timeout errors and accessing status codes in golang

I have some code (see below) written in Go which is supposed to "fan-out" HTTP requests, and collate/aggregate the details back.

I'm new to golang and so expect me to be a nOOb and my knowledge to be limited

The output of the program is currently something like:

{
    "Status":"success",
    "Components":[
        {"Id":"foo","Status":200,"Body":"..."},
        {"Id":"bar","Status":200,"Body":"..."}, 
        {"Id":"baz","Status":404,"Body":"..."}, 
        ...
    ]
}

There is a local server running that is purposely slow (sleeps for 5 seconds and then returns a response). But I have other sites listed (see code below) that sometime trigger an error as well (if they error, then that's fine).

The problem I have at the moment is how best to handle these errors, and specifically the "timeout" related errors; in that I'm not sure how to recognise if a failure is a timeout or some other error?

At the moment I get a blanket error back all the time:

Get http://localhost:8080/pugs: read tcp 127.0.0.1:8080: use of closed network connection

Where http://localhost:8080/pugs will generally be the url that failed (hopefully by timeout!). But as you can see from the code (below), I'm not sure how to determine the error code is related to a timeout nor how to access the status code of the response (I'm currently just blanket setting it to 404 but obviously that's not right - if the server was to error I'd expect something like a 500 status code and obviously I'd like to reflect that in the aggregated response I send back).

The full code can be seen below. Any help appreciated.

    package main

    import (
            "encoding/json"
            "fmt"
            "io/ioutil"
            "net/http"
            "sync"
            "time"
    )

    type Component struct {
            Id  string `json:"id"`
            Url string `json:"url"`
    }

    type ComponentsList struct {
            Components []Component `json:"components"`
    }

    type ComponentResponse struct {
            Id     string
            Status int
            Body   string
    }

    type Result struct {
            Status     string
            Components []ComponentResponse
    }

    var overallStatus string = "success"

    func main() {
            var cr []ComponentResponse
            var c ComponentsList

            b := []byte(`{"components":[{"id":"local","url":"http://localhost:8080/pugs"},{"id":"google","url":"http://google.com/"},{"id":"integralist","url":"http://integralist.co.uk/"},{"id":"sloooow","url":"http://stevesouders.com/cuzillion/?c0=hj1hfff30_5_f&t=1439194716962"}]}`)

            json.Unmarshal(b, &c)

            var wg sync.WaitGroup

            timeout := time.Duration(1 * time.Second)
            client := http.Client{
                    Timeout: timeout,
            }

            for i, v := range c.Components {
                    wg.Add(1)

                    go func(i int, v Component) {
                            defer wg.Done()

                            resp, err := client.Get(v.Url)

                            if err != nil {
                                fmt.Printf("Problem getting the response: %s\n", err)

                                cr = append(cr, ComponentResponse{
                                    v.Id,
                                    404,
                                    err.Error(),
                                })
                            } else {
                                    defer resp.Body.Close()
                                    contents, err := ioutil.ReadAll(resp.Body)
                                    if err != nil {
                                            fmt.Printf("Problem reading the body: %s\n", err)
                                    }

                                    cr = append(cr, ComponentResponse{
                                            v.Id,
                                            resp.StatusCode,
                                            string(contents),
                                    })
                            }
                    }(i, v)
            }
            wg.Wait()

            j, err := json.Marshal(Result{overallStatus, cr})
            if err != nil {
                    fmt.Printf("Problem converting to JSON: %s\n", err)
                    return
            }

            fmt.Println(string(j))
    }

Upvotes: 6

Views: 6883

Answers (3)

The Fool
The Fool

Reputation: 20420

I am adding this for completes, as the correct answer was provided by Dave C in the comments of the accepted answer.

We can check if the error is or contains a net.Error and check if it is a timeout.

resp, err := client.Get(url)
if err != nil {
    // if there is an error check if its a timeout error
    var netErr net.Error
    if errors.As(err, &netErr) && netErr.Timeout() {
        // handle timeout
           return
    } 
    // otherwise handle other types of error
}

Upvotes: 3

Integralist
Integralist

Reputation: 6165

The Go 1.5 release solved this issue by being more specific about the type of error it has handled.

So if you see this example https://github.com/Integralist/Go-Requester/blob/master/requester.go#L38 you'll see that I'm able to apply a regex pattern to the error message to decipher if the error was indeed a timeout or not

status := checkError(err.Error())

func checkError(msg string) int {
    timeout, _ := regexp.MatchString("Timeout", msg)

    if timeout {
        return 408
    }

  return 500
}

Upvotes: -4

golliher
golliher

Reputation: 1417

If you want to fan out then aggregate results and you want specific timeout behavior the net/http package isn't giving you, then you may want to use goroutines and channels.

I just watched this video today and it will walk you through exactly those scenarios using the concurrency features of Go. Plus, the speaker Rob Pike is quite the authority -- he explains it much better than I could.

https://www.youtube.com/watch?v=f6kdp27TYZs

Upvotes: 1

Related Questions