Problems testing spring webflux Webclient with high load

Question

I am trying to learn Spring Webflux comming from C# and NetCore, we have a very similar problem like this post, where a third party service provider has some response time problems.

But testing with spring-webclient is doubling the response time, I do not know if I am missing something

I tried to create a similar example with:

• A computer running 3 servers
  • Demo server that just simulates some random delay time (port 8080)
  • Test Server in C# using async to call my "Wait" server (port 5000)
  • Test Server with spring and webclient to call my "Wait" server (port 8081)
• Other computer running JMeter with 1000 clients and 10 rounds each one

Some code

Wait server

Just a simple route

@Configuration
class TestRouter(private val middlemanDemo: MiddlemanDemo) {

    @Bean
    fun route() = router {
        GET("/testWait", middlemanDemo::middleTestAndGetWait)
    }
}

The handler has a Random generator with a seed, so each test can generate the same sequence of delays

@Service
class TestWaiter {

    companion object RandomManager {

        private lateinit var random: Random

        init {
            resetTimer()
        }

        @Synchronized
        fun next(): Long {
            val random = random.nextLong(0, 10)
            return random * 2
        }

        fun resetTimer() {
            random = Random(12345)
        }
    }

    private val logger = LoggerFactory.getLogger(javaClass)

    fun testAndGetWait(request: ServerRequest): Mono<ServerResponse> {
        val wait = next()
        logger.debug("Wait is: {}", wait)
        return ServerResponse
                .ok()
                .json()
                .bodyValue(wait)
                .delayElement(Duration.ofSeconds(wait))
    }

    fun reset(request: ServerRequest): Mono<ServerResponse> {
        logger.info("Random reset")
        resetTimer()
        return ServerResponse
                .ok()
                .build()
    }
}

Load testing the server with JMeter I can see a steady response time of around 9-10 seconds and a max throughput of 100/sec:

C# async Demo server

Trying a middle man with C#, this server just calls the main demo server:

The controller

[HttpGet]
public async Task<string> Get()
{
    return await _waiterClient.GetWait();
}

And the service with the httpClient

    private readonly HttpClient _client;

    public WaiterClient(HttpClient client)
    {
        _client = client;
        client.BaseAddress = new Uri("http://192.168.0.121:8080");
    }

    public async Task<string> GetWait()
    {
        var response = await _client.GetAsync("/testWait");
        var waitTime = await response.Content.ReadAsStringAsync();
        return waitTime;
    }
}

Testing this service gives the same response time, with a little less throughput for the overhead, but it is understandable

The spring-webclient implementation

This client is also really simple, just one route

@Configuration
class TestRouter(private val middlemanDemo: MiddlemanDemo) {

    @Bean
    fun route() = router {
        GET("/testWait", middlemanDemo::middleTestAndGetWait)
    }
}

The handler just calls the service using the webclient

@Service
class MiddlemanDemo {

    private val client = WebClient.create("http://127.0.0.1:8080")

    fun middleTestAndGetWait(request: ServerRequest): Mono<ServerResponse> {
        return client
                .get()
                .uri("/testWait")
                .retrieve()
                .bodyToMono(Int::class.java)
                .flatMap(::processResponse)
    }

    fun processResponse(delay: Int): Mono<ServerResponse> {
        return ServerResponse
                .ok()
                .bodyValue(delay)
    }
}

However, running the tests, the throughput only get to 50/sec

And the response time doubles like if I had another wait, until the load goes down again

Answer 1

I am marking KL.Lee answer because it pointed me in the right way, but I will add the complete solution for anyone to find:

The key was to create a connection pool according to my needs. The default is 500 as JK.Lee mentioned.

@Service
class MiddlemanDemo(webClientBuilder: WebClient.Builder) {

    private val client: WebClient

    init {
        val provider = ConnectionProvider.builder("fixed")
                .maxConnections(2000) // This is the important part
                .build()
        val httpClient = HttpClient
                .create(provider)
        client = webClientBuilder
                .clientConnector(ReactorClientHttpConnector(httpClient))
                .baseUrl("http://localhost:8080")
                .build()
    }

    fun middleTestAndGetWait(request: ServerRequest): Mono<ServerResponse> {

        return client
                .get()
                .uri("/testWait")
                .retrieve()
                .bodyToMono(Int::class.java)
                .flatMap(::processResponse)
    }

    fun processResponse(delay: Int): Mono<ServerResponse> {
        return ServerResponse
                .ok()
                .bodyValue(delay)
    }
}

Answer 2

I think it may be caused by pool acquire time.

I assume your server gets over 1k TPS and each request looks to take about 9 seconds. But the default HTTP client connection pool is 500. Please refer to Projector Reactor - Connection Pool.

Please check the logs have PoolAcquireTimeoutException or whether your server takes some time to wait pool acquisition.