Too many open files error android

Question

I'm implementing a service for uploading files to a cloud service. I'm uploading huge files in chunks, using OkHttp. The logic is like this, I get the loading url and headers from the server, and then upload the chunk to the cloud, and so on. The problem that after about ~200 chunks I get this error:

 W/System.err: java.net.ConnectException: failed to connect to piboxdev.blob.core.windows.net/40.118.73.216 (port 443) after 10000ms: connect failed: EMFILE (Too many open files)
 04-07 11:10:55.963 4945-5653/? W/StreamManager: Dropping non-bitmap icon from notification.
 04-07 11:10:55.977 13347-15602/by.set.pibox W/System.err:     at libcore.io.IoBridge.connect(IoBridge.java:124)
 04-07 11:10:55.977 13347-15602/by.set.pibox W/System.err:     at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:183)
 04-07 11:10:55.977 13347-15602/by.set.pibox W/System.err:     at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:456)
 04-07 11:10:55.977 13347-15602/by.set.pibox W/System.err:     at java.net.Socket.connect(Socket.java:882)
 04-07 11:10:55.977 13347-15602/by.set.pibox W/System.err:     at com.squareup.okhttp.internal.Platform$Android.connectSocket(Platform.java:190)
 04-07 11:10:55.977 13347-15602/by.set.pibox W/System.err:     at com.squareup.okhttp.Connection.connectSocket(Connection.java:196)
 04-07 11:10:55.977 13347-15602/by.set.pibox W/System.err:     at com.squareup.okhttp.Connection.connect(Connection.java:172)
 04-07 11:10:55.977 13347-15602/by.set.pibox W/System.err:     at com.squareup.okhttp.Connection.connectAndSetOwner(Connection.java:367)
 04-07 11:10:55.978 13347-15602/by.set.pibox W/System.err:     at com.squareup.okhttp.OkHttpClient$1.connectAndSetOwner(OkHttpClient.java:128)
 04-07 11:10:55.978 13347-15602/by.set.pibox W/System.err:     at com.squareup.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:328)
 04-07 11:10:55.978 13347-15602/by.set.pibox W/System.err:     at com.squareup.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:245)
 04-07 11:10:55.978 13347-15602/by.set.pibox W/System.err:     at com.squareup.okhttp.Call.getResponse(Call.java:267)
 04-07 11:10:55.978 13347-15602/by.set.pibox W/System.err:     at com.squareup.okhttp.Call$ApplicationInterceptorChain.proceed(Call.java:224)
 04-07 11:10:55.978 13347-15602/by.set.pibox W/System.err:     at com.squareup.okhttp.Call.getResponseWithInterceptorChain(Call.java:195)
 04-07 11:10:55.978 13347-15602/by.set.pibox W/System.err:     at com.squareup.okhttp.Call.execute(Call.java:79)
 04-07 11:10:55.978 13347-15602/by.set.pibox W/System.err:     at by.set.pibox.upload.uploadService.HttpUploadTask.loadToAzure(HttpUploadTask.java:254)

Here is the code which handles the upload:

   try {
        HashMap<String, String> headers = response.getHeaders();
        String url = response.getUrl();

        if (source==null)
            source = Okio.buffer(Okio.source(file));

        if (content==null)
        content = new ChunkFileRequestBody(source, size, contentType);

        Request.Builder builder = new Request.Builder();

        for (Map.Entry<String, String>entry : headers.entrySet()) {
          builder.addHeader(entry.getKey(), entry.getValue());
        }

        Request azureRequest =  builder.url(url).put(content).build();
        Response azureResponse = client.newCall(azureRequest).execute();
        if (azureResponse.code()/100==2 && shouldContinue) {
            Log.e(getClass().getSimpleName(), azureResponse.code() + " " + azureResponse.message());
            uploadedBodyBytes +=size;
            if (uploadedBodyBytes>=totalBodyBytes) {
                cleanUp();
                confirmUpload();
            }
            else {
                broadcastProgress(uploadedBodyBytes, totalBodyBytes);
                upload();
            }

And this is the custom content body:

public class ChunkFileRequestBody extends RequestBody {

private final String contentType;
private long uploadSize;
private BufferedSource source;

public ChunkFileRequestBody(BufferedSource source, Long size, String contentType) {
    this.source = source;
    this.contentType = contentType;
    this.uploadSize=size;
}

@Override
public long contentLength() {
    return uploadSize;
}

@Override
public MediaType contentType() {
    return MediaType.parse(contentType);
}

@Override
public void writeTo(BufferedSink sink) throws IOException {
        source.readFully(sink.buffer(), uploadSize);
        sink.flush();
}


}

This can't be the leak in file handling, since I open the file only once. The memory usage seems to be fine, as well. It seems almost like OkHttp fails to close the sockets properly. I asked the authors of the library, but they couldn't help with the case.

Did somebody ever got something like this? Please help

Answer 1

How are you instantiating your http client? I just resolved this exact issue by not instantiating a new client every time, rather creating just one and reusing it.