HttpClient request succeeds with timeout defined, but hangs without

Question

I found a strange hack-y way to fix my code and I was wondering if anyone could explain why it works. I am writing code that communicates with a REST API to upload a video file split into multiple HTTP requests.

I was having a problem with one of my video part requests connecting, but never responding. The program uploads the video in five parts, but it would always hang on the third part of the five parts. I decided to add a request hard timeout to force the program to skip that hanging part. Well, magically after adding that timer, there is no more hangup!

Any ideas why this is the case? The request doesn't actually timeout, yet the addition of this code keeps my program chugging.

private void uploadParts(String assetId) throws IOException {
    //set up post request
    HttpClient client = HttpClientBuilder.create().build();

    String url = "";

    //prepare video
    File video = new File("files/video.mp4");
    BufferedInputStream bis = new BufferedInputStream(new FileInputStream(video));

    int partMaxSize = 1024 * 1024 * 5;
    byte[] buffer = new byte[partMaxSize];
    double fileSize = video.length();
    System.out.println(fileSize);
    System.out.println(fileSize / partMaxSize);
    int parts = (int) Math.ceil(fileSize / partMaxSize);
    System.out.println(parts);

    for(int i = 1; i < parts+1; i++)    {
        String partNumber = i + "";
        System.out.println("part: " + partNumber);
        int partSize = (int) (i < parts ? partMaxSize : fileSize);
        fileSize -= partSize;
        int tmp = 0;
        tmp = bis.read(buffer);
        url = String.format("https://www.site.com/upload/multipart/%s/%s", assetId, partNumber);

        final HttpPut request = new HttpPut(url);
        request.addHeader("Authorization", "Bearer " + accessToken);
        request.addHeader("Content-Type", "application/octet-stream");
        request.setEntity(new ByteArrayEntity(buffer));

        //Magical code start
        int hardTimeout = 5; // seconds
        TimerTask task = new TimerTask() {
            @Override
            public void run() {
                if (request != null) {
                    request.abort();
                }
            }
        };
        new Timer(true).schedule(task, hardTimeout * 1000);
        //Magical code end

        HttpResponse response = client.execute(request);
        System.out.println(response.getStatusLine().getReasonPhrase());
    }
    bis.close();
}

If i leave out the magical code section, my code hangs on the third part. If I include it, the program runs through fine.

Answer 1

I found the answer! Turns out HttpClient only allows a certain number of connections at a time. According to my code, the default maximum connections is 2. I needed to close each connection after they were complete and the upload ran fine.

Fixed code adds request connection release.

private void uploadParts(String assetId) throws IOException {
    //set up post request
    HttpClient client = HttpClientBuilder.create().build();

    String url = "";

    //prepare video
    File video = new File("files/video.mp4");
    BufferedInputStream bis = new BufferedInputStream(new FileInputStream(video));

    int partMaxSize = 1024 * 1024 * 5;
    byte[] buffer = new byte[partMaxSize];
    double fileSize = video.length();
    System.out.println(fileSize);
    System.out.println(fileSize / partMaxSize);
    int parts = (int) Math.ceil(fileSize / partMaxSize);
    System.out.println(parts);

    for(int i = 1; i < parts+1; i++)    {
        String partNumber = i + "";
        System.out.println("part: " + partNumber);
        int partSize = (int) (i < parts ? partMaxSize : fileSize);
        fileSize -= partSize;
        int tmp = 0;
        tmp = bis.read(buffer);
        url = String.format("https://www.site.com/upload/multipart/%s/%s", assetId, partNumber);

        final HttpPut request = new HttpPut(url);
        request.addHeader("Authorization", "Bearer " + accessToken);
        request.addHeader("Content-Type", "application/octet-stream");
        request.setEntity(new ByteArrayEntity(buffer));

        //Magical code start
        int hardTimeout = 5; // seconds
        TimerTask task = new TimerTask() {
            @Override
            public void run() {
                if (request != null) {
                    request.abort();
                }
            }
        };
        new Timer(true).schedule(task, hardTimeout * 1000);
        //Magical code end

        HttpResponse response = client.execute(request);
        request.releaseConnection();
        System.out.println(response.getStatusLine().getReasonPhrase());
    }
    bis.close();
}

The timer was working because it was closing my old connections after 10 seconds. Thank you for your input, guys.