how to close/abort a Golang http.Client POST prematurely

Question

I'm using http.Client for the client-side implementation of a long-poll:

resp, err := client.Post(url, "application/json", bytes.NewBuffer(jsonPostBytes))
if err != nil {
    panic(err)
}
defer resp.Body.Close()

var results []*ResponseMessage
err = json.NewDecoder(resp.Body).Decode(&results)  // code blocks here on long-poll

Is there a standard way to pre-empt/cancel the request from the client-side?

I imagine that calling resp.Body.Close() would do it, but I'd have to call that from another goroutine, as the client is normally already blocked in reading the response of the long-poll.

I know that there is a way to set a timeout via http.Transport, but my app logic need to do the cancellation based on a user action, not just a timeout.

Answer 1

To add to the other answers that attach context.Context to http requests, since 1.13 we have:

A new function NewRequestWithContext has been added and it accepts a Context that controls the entire lifetime of the created outgoing Request, suitable for use with Client.Do and Transport.RoundTrip.

https://golang.org/doc/go1.13#net/http

This function can be used instead of using NewRequest and then Request.WithContext.

req, err := http.NewRequest(...)
if err != nil {...}
req.WithContext(ctx)

becomes

req, err := http.NewRequestWithContext(ctx, ...)
if err != nil {...}

Answer 2

@Paulo Casaretto 's answer is right, should using http.Request.WithContext.

Here is a full demo (be aware of the time numbers: 5, 10, 30 seconds).

HTTP Server:

package main

import (
    "fmt"
    "log"
    "net/http"
    "time"
)

func handler(w http.ResponseWriter, r *http.Request) {
    fmt.Println("before sleep")
    time.Sleep(10 * time.Second)
    fmt.Println("after sleep")

    fmt.Fprintf(w, "Hi")
}

func main() {
    http.HandleFunc("/", handler)
    log.Fatal(http.ListenAndServe(":9191", nil))
}

The HTTP Server console print:

before sleep
after sleep 

HTTP Client:

package main

import (
    "context"
    "fmt"
    "net/http"
    "time"
)

func main() {
    ctx, cancel := context.WithCancel(context.Background())

    go func() {
        fmt.Println("before request")
        client := &http.Client{Timeout: 30 * time.Second}
        req, err := http.NewRequest("GET", "http://127.0.0.1:9191", nil)
        if err != nil {
            panic(err)
        }
        req = req.WithContext(ctx)
        _, err = client.Do(req)
        if err != nil {
            panic(err)
        }
        fmt.Println("will not reach here")
    }()

    time.Sleep(5 * time.Second)
    cancel()
    fmt.Println("finished")
}

The HTTP Client console print:

before request
finished

Answer 3

The standard way is to use a context of type context.Context and pass it around to all the functions that need to know when the request is cancelled.

func httpDo(ctx context.Context, req *http.Request, f func(*http.Response, error) error) error {
    // Run the HTTP request in a goroutine and pass the response to f.
    tr := &http.Transport{}
    client := &http.Client{Transport: tr}
    c := make(chan error, 1)
    go func() { c <- f(client.Do(req)) }()
    select {
    case <-ctx.Done():
        tr.CancelRequest(req)
        <-c // Wait for f to return.
        return ctx.Err()
    case err := <-c:
        return err
    }
}

golang.org/x/net/context

// A Context carries a deadline, cancelation signal, and request-scoped values
// across API boundaries. Its methods are safe for simultaneous use by multiple
// goroutines.
type Context interface {
    // Done returns a channel that is closed when this Context is canceled
    // or times out.
    Done() <-chan struct{}

    // Err indicates why this context was canceled, after the Done channel
    // is closed.
    Err() error

    // Deadline returns the time when this Context will be canceled, if any.
    Deadline() (deadline time.Time, ok bool)

    // Value returns the value associated with key or nil if none.
    Value(key interface{}) interface{}
}

Source and more on https://blog.golang.org/context

Update

As Paulo mentioned, Request.Cancel is now deprecated and the author should pass the context to the request itself(using *Request.WithContext) and use the cancellation channel of the context(to cancel the request).

package main

import (
    "context"
    "net/http"
    "time"
)

func main() {
    cx, cancel := context.WithCancel(context.Background())
    req, _ := http.NewRequest("GET", "http://google.com", nil)
    req = req.WithContext(cx)
    ch := make(chan error)

    go func() {
        _, err := http.DefaultClient.Do(req)
        select {
        case <-cx.Done():
            // Already timedout
        default:
            ch <- err
        }
    }()

    // Simulating user cancel request
    go func() {
        time.Sleep(100 * time.Millisecond)
        cancel()
    }()
    select {
    case err := <-ch:
        if err != nil {
            // HTTP error
            panic(err)
        }
        print("no error")
    case <-cx.Done():
        panic(cx.Err())
    }

}

Answer 4

Using CancelRequest is now deprecated.

The current strategy is to use http.Request.WithContext passing a context with a deadline or that will be canceled otherwise. Just use it like a normal request afterwards.

req, err := http.NewRequest("GET", "http://example.com", nil)
// ...
req.Header.Add("If-None-Match", `W/"wyzzy"`)
req = req.WithContext(ctx)
resp, err := client.Do(req)
// ...

Answer 5

Nope, client.Post is a handy wrapper for 90% of use-cases where request cancellation is not needed.

Probably it will be enough simply to reimplement your client to get access to underlying Transport object, which has CancelRequest() function.

Just a quick example:

package main

import (
    "log"
    "net/http"
    "time"
)

func main() {
    req, _ := http.NewRequest("GET", "http://google.com", nil)
    tr := &http.Transport{} // TODO: copy defaults from http.DefaultTransport
    client := &http.Client{Transport: tr}
    c := make(chan error, 1)
    go func() {
        resp, err := client.Do(req)
        // handle response ...
        _ = resp
        c <- err
    }()

    // Simulating user cancel request channel
    user := make(chan struct{}, 0)
    go func() {
        time.Sleep(100 * time.Millisecond)
        user <- struct{}{}
    }()

    for {
        select {
        case <-user:
            log.Println("Cancelling request")
            tr.CancelRequest(req)
        case err := <-c:
            log.Println("Client finished:", err)
            return
        }
    }
}