Simple golang http rest service hangs under load

Question

I am trying to test how golang can handle big loads to compare it with our current applications made with Java.

What I did is a simple echo rest service like that (I am adding just the important parts of my code):

// Return default message for root routing
func Index(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Hello, %q", html.EscapeString(r.URL.Path))
}

// Main function
func main() {

    router := mux.NewRouter() //.StrictSlash(true)
    router.HandleFunc("/", Index).Methods("GET")
    router.HandleFunc("/echo/{message}", echoHandler(calledServiceURL)).Methods("GET")

    log.Println("Running server....")

    log.Fatal(http.ListenAndServe(port, router))
}

I did a test by using ab tool and it worked well with -c 500 -n 500 but when I tried to test with a big load like this

ab -c 500 -n 50000 http://localhost:9596/echo/javier

The process works well for a couple of seconds but then seems it close the tcp connection as I receive the following error:

Benchmarking localhost (be patient)
apr_socket_recv: Connection reset by peer (54)
Total of 501 requests completed

Is is due to a OS limitations that my test reached or is it the limit that my golang app could handle?

Is there a better way to process requests and then avoid the program to close connections? (queue requests or something like that).

Thanks in advance J

Answer 1

Do you happen to use OSX? I understand that ab is broken on OSX.

Another thing you could try is to use -k to use the keep alive flag, but that may not be something that you want.

50000 is also close to the maximum number of sockets on an interface, so maybe your sockets are exhausted. Sockets are not directly reusable, because they will be in TIME_WAIT state for a minute or two. Exact value can vary per OS and configuration.

However, the code looks fine too me.

The following code worked fine for me:

package main

import (
    "fmt"
    "github.com/gorilla/mux"
    "log"
    "net/http"
)


func Echo(w http.ResponseWriter, r *http.Request) {
    v := mux.Vars(r)

    fmt.Fprintf(w, "Echo %v", v["message"])
}


func main() {

    router := mux.NewRouter() //.StrictSlash(true)
    router.HandleFunc("/echo/{message}", Echo).Methods("GET")

    log.Println("Running server....")

    log.Fatal(http.ListenAndServe("localhost:8080", router))
}

And gives the following results:

ab -c 500 -n 50000 localhost:8080/echo/foobar
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 5000 requests
Completed 10000 requests
Completed 15000 requests
Completed 20000 requests
Completed 25000 requests
Completed 30000 requests
Completed 35000 requests
Completed 40000 requests
Completed 45000 requests
Completed 50000 requests
Finished 50000 requests


Server Software:        
Server Hostname:        localhost
Server Port:            8080

Document Path:          /echo/foobar
Document Length:        12 bytes

Concurrency Level:      500
Time taken for tests:   2.471 seconds
Complete requests:      50000
Failed requests:        0
Total transferred:      6450000 bytes
HTML transferred:       600000 bytes
Requests per second:    20233.39 [#/sec] (mean)
Time per request:       24.712 [ms] (mean)
Time per request:       0.049 [ms] (mean, across all concurrent requests)
Transfer rate:          2548.93 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   18 122.8      3    1034
Processing:     0    5  14.9      4     225
Waiting:        0    4  14.7      3     222
Total:          1   24 132.7      6    1245

Percentage of the requests served within a certain time (ms)
  50%      6
  66%      7
  75%      7
  80%      7
  90%     12
  95%     20
  98%     30
  99%   1040
 100%   1245 (longest request)

This is executed on Ubuntu 18.04.