How do you rate limit a Golang server hosted on Cloud Run?

Question

Since Cloud run is "stateless", I'm assuming state doesn't persist between requests, so creating a map of ip addresses would not work. Or would something like limiter work?

Answer 1

A request is handled by a request handler which is request scoped, whereas your limiter has a global scope.

Let me illustrate that with some code. We have the request scoped variable i and the globally scoped variable j. Furthermore, we have a global limiter.

So there is exactly one instance of the limiter and j, but for reach request a variable named i is created and distinct for that request.

package main

import (
    "flag"
    "fmt"
    "log"
    "net/http"
    "sync/atomic"
    "time"

    "github.com/ulule/limiter/v3"
    "github.com/ulule/limiter/v3/drivers/middleware/stdlib"
    "github.com/ulule/limiter/v3/drivers/store/memory"
)

var bind string

func init() {
    // Make the bind address configurable
    flag.StringVar(&bind, "bind", ":9090", "address to bind to")
}

func main() {
    flag.Parse()

    // The rate you want to employ
    // We use unusual values here for testing purposes
    rate := limiter.Rate{
        Period: 5 * time.Second,
        Limit:  1,
    }

    // We use an in-memory store for the sake of simplicity.
    // Furthermore, as a security measure, persistence might introduce
    // an unneccessary complexity as well as a point of attack itself
    // by overloading the persistence mechanism.
    l := limiter.New(memory.NewStore(), rate)

    middleware := stdlib.NewMiddleware(l)

    // for further clarification, we add a globally scoped counter
    var j uint64

    // We tell the http server to take requests to /hello...
    http.Handle("/hello",
        // put them through or globally scoped middleware
        // which will enfore the rate limit and...
        middleware.Handler(

            // executes the actual http.Handler if the limit is not reached.
            http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {

                // request scoped...
                var i int
                // so i will always be 1 after increment
                i++

                // But we also increment our globally scoped j.
                // Since multiple goroutines might access j simultaneously, we need
                // to take the precaution of an atomic operation.
                atomic.AddUint64(&j, 1)

                w.Write([]byte(
                    fmt.Sprintf("Hello, world!\nrequest scoped i: %d, global scoped j:%d\n", i, atomic.LoadUint64(&j))))
            })))

    // Last but not least we start the server
    log.Printf("Starting server bound to '%s'", bind)
    log.Fatal(http.ListenAndServe(bind, nil))
}

Now, when we run this code and call the URL with curl, we get a response (limit did not kick in) and both i and j have a value of 1.

$ curl -iv --no-keepalive http://localhost:9090/hello
*   Trying ::1:9090...
* Connected to localhost (::1) port 9090 (#0)
> GET /hello HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.69.1
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< X-Ratelimit-Limit: 1
X-Ratelimit-Limit: 1
< X-Ratelimit-Remaining: 0
X-Ratelimit-Remaining: 0
< X-Ratelimit-Reset: 1588596822
X-Ratelimit-Reset: 1588596822
< Date: Mon, 04 May 2020 12:53:37 GMT
Date: Mon, 04 May 2020 12:53:37 GMT
< Content-Length: 53
Content-Length: 53
< Content-Type: text/plain; charset=utf-8
Content-Type: text/plain; charset=utf-8

< 
Hello, world!
request scoped i: 1, global scoped j:1
* Connection #0 to host localhost left intact

If we call the URL again within 5 seconds, the rate limiter kicks in, and denies us access:

$ curl -iv --no-keepalive http://localhost:9090/hello
*   Trying ::1:9090...
* Connected to localhost (::1) port 9090 (#0)
> GET /hello HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.69.1
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 429 Too Many Requests
HTTP/1.1 429 Too Many Requests
< Content-Type: text/plain; charset=utf-8
Content-Type: text/plain; charset=utf-8
< X-Content-Type-Options: nosniff
X-Content-Type-Options: nosniff
< X-Ratelimit-Limit: 1
X-Ratelimit-Limit: 1
< X-Ratelimit-Remaining: 0
X-Ratelimit-Remaining: 0
< X-Ratelimit-Reset: 1588596822
X-Ratelimit-Reset: 1588596822
< Date: Mon, 04 May 2020 12:53:38 GMT
Date: Mon, 04 May 2020 12:53:38 GMT
< Content-Length: 15
Content-Length: 15

< 
Limit exceeded
* Connection #0 to host localhost left intact

And, after several seconds of waiting, we call the URL again, the globally scoped variable increments, whereas the request scoped variable again is 1:

$ curl -iv --no-keepalive http://localhost:9090/hello
*   Trying ::1:9090...
* Connected to localhost (::1) port 9090 (#0)
> GET /hello HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.69.1
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
HTTP/1.1 200 OK
< X-Ratelimit-Limit: 1
X-Ratelimit-Limit: 1
< X-Ratelimit-Remaining: 0
X-Ratelimit-Remaining: 0
< X-Ratelimit-Reset: 1588596884
X-Ratelimit-Reset: 1588596884
< Date: Mon, 04 May 2020 12:54:39 GMT
Date: Mon, 04 May 2020 12:54:39 GMT
< Content-Length: 53
Content-Length: 53
< Content-Type: text/plain; charset=utf-8
Content-Type: text/plain; charset=utf-8

< 
Hello, world!
request scoped i: 1, global scoped j:2
* Connection #0 to host localhost left intac