Slow Apache httpclient 4.1 compared to JMeter

Question

I have a simple 1 thread loop using Apache HttpClient 4.1. It connects to my Apache httpd web server on localhost.

I'm averaging 2.5 ms per request/response. JMeter, on the other hand, averages 1 ms. (Apache Benchmark, ab, does it in 0.4ms but since that is native code perhaps there is no comparison.)

The code is just:

    final HttpGet httpGet = new HttpGet(testUrl);
    while (true) {
        try {
            final long startNanoTime = System.nanoTime();
            final HttpResponse httpResponse = httpClient.execute(httpGet); 
            final InputStream inputStream = httpResponse.getEntity().getContent();

            final byte[] buffer = new byte[8192];
            int size = inputStream.read(buffer);
            while (size > 0) {
                size = inputStream.read(buffer);
            }

                            // Elapsed time measured here
            final long elapsed = System.nanoTime() - startNanoTime;

            inputStream.close();

        } catch (MalformedURLException e) {
            // Should never happen
            throw new RuntimeException(e);
        } catch (IOException e) {
            // Count
            errors++;
            throw new RuntimeException(e);
        }
    }

Answer 1

My testing shows otherwise, take stackoverflow.com GET request for example over 10 repetitions :

As you may see (or not) from the image the average time is 712ms when using jmeter with View Results in a Table. Notice that this listener doesn't print out the response body just the request stats.

And here is my code from Java :

public static void main(String[] args) throws Exception{        
        long totalTimeMS = 0;

        for (int i = 0; i < 10; i++) {

        long startTime = System.currentTimeMillis();


        HttpGet get = new HttpGet("http://stackoverflow.com");
        HttpClient client = new DefaultHttpClient();
        client.execute(get);       

        long endTime = System.currentTimeMillis();
        long duration = (endTime-startTime);
        totalTimeMS +=duration;
        System.out.format("Duration %d ms\n", duration);
        }

        System.out.format("Average time is %d ms", (totalTimeMS/10));
    }

So I do not care about the response body as well. But here are the results (lot faster) :

Duration 615 ms
Duration 263 ms
Duration 264 ms
Duration 262 ms
Duration 268 ms
Duration 266 ms
Duration 265 ms
Duration 266 ms
Duration 268 ms
Duration 263 ms
Average time is 300 ms

Now another case in jmeter when using View Results in a Tree when you can actually see the response body plus the View Results in a Table because we still need the time.

I'm not going to attach screenshot because the answer will become less readable but the average time this time is 812 ms so about 100ms more than previously.

Now the java code that cares about the response body (new method) :

public static String convertStreamToString(InputStream is) throws IOException {
        if (is != null) {
            StringBuilder sb = new StringBuilder();
            String line;
            try {
                BufferedReader reader = new BufferedReader(new InputStreamReader(is, "UTF-8"));
                while ((line = reader.readLine()) != null) {
                    sb.append(line).append("\n");
                }
            } finally {
                is.close();
            }
            return sb.toString();
        } else {
            return "";
        }
    }

And I've modified slightly the previous code so it prints out the response, simulating the jmeter behavior, posting relevant part :

HttpGet get = new HttpGet("http://stackoverflow.com");
        HttpClient client = new DefaultHttpClient();
        HttpResponse response = client.execute(get);       

        long endTime = System.currentTimeMillis();
        long duration = (endTime-startTime);
        totalTimeMS +=duration;
        System.out.println(convertStreamToString(response.getEntity().getContent()));
        System.out.format("Duration %d ms\n", duration);

Results are following :

Duration 678 ms  + (including printing of response body)
Duration 264 ms + (including printing of response body)
Duration 269 ms + (including printing of response body)
Duration 262 ms + (including printing of response body)
Duration 263 ms + (including printing of response body)
Duration 265 ms + (including printing of response body)
Duration 262 ms + (including printing of response body)
Duration 267 ms + (including printing of response body)
Duration 264 ms + (including printing of response body)
Duration 264 ms + (including printing of response body)
Average time is 305 ms

Reponse time went up by 5 ms. So I don't know how jmeter got to be faster than just plain java code. Anyhow jmeter is really great tool, one of the best one arround (for free).