CODEWITHSUNDEEP
javajcloudscloudfilesrackspace-cloudfiles
oak
oak

Reputation: 3036

jclouds IOExpection: Error writing request body to server

We are using jclouds with Rackspace and when uploading lots of files via cloudfile api (multi threaded)

Once in while we are getting an exception on objectApi.put line (see example code at bottom)

Exception

16-Jul-2015 11:58:00.811 SEVERE [threadsPool-1]      org.jclouds.logging.jdk.JDKLogger.logError error after writing 8192/streaming bytes to https://*****/****.jpg
 java.io.IOException: Error writing request body to server
        at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.checkError(HttpURLConnection.java:3478)
        at sun.net.www.protocol.http.HttpURLConnection$StreamingOutputStream.write(HttpURLConnection.java:3461)
        at com.google.common.io.CountingOutputStream.write(CountingOutputStream.java:53)
        at com.google.common.io.ByteStreams.copy(ByteStreams.java:74)
        at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.writePayloadToConnection(JavaUrlHttpCommandExecutorService.java:297)
        at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:160)
        at org.jclouds.http.internal.JavaUrlHttpCommandExecutorService.convert(JavaUrlHttpCommandExecutorService.java:64)
        at org.jclouds.http.internal.BaseHttpCommandExecutorService.invoke(BaseHttpCommandExecutorService.java:91)
        at org.jclouds.rest.internal.InvokeHttpMethod.invoke(InvokeHttpMethod.java:90)
        at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:73)
        at org.jclouds.rest.internal.InvokeHttpMethod.apply(InvokeHttpMethod.java:44)
        at org.jclouds.reflect.FunctionalReflection$FunctionalInvocationHandler.handleInvocation(FunctionalReflection.java:117)
        at com.google.common.reflect.AbstractInvocationHandler.invoke(AbstractInvocationHandler.java:87)
        at com.sun.proxy.$Proxy176.put(Unknown Source)
        at

Similar issue with S3

can be found here

Example Code

ObjectApi objectApi = cloudFiles.getObjectApi(REGION, container);
ByteSource byteSource = Files.asByteSource(file);
Payload payload = Payloads.newByteSourcePayload(byteSource);
objectApi.put(hashedName, payload);

The question:

Any one has experience some behavior like that? maybe someone has workaround for that kind of issue?

Thanks

Alon

Upvotes: 1

Views: 3652

Answers (1)

zacksh
zacksh

Reputation: 316

Networks are unreliable, so expect some exceptions when using cloud services, especially when dealing with many files. Specifically for jclouds uploads, we have some example code here:

https://github.com/jclouds/jclouds-examples/tree/master/blobstore-uploader

Edit: I have also added a JIRA issue to make sure we add a test specifically for this situation in swift:

https://issues.apache.org/jira/browse/JCLOUDS-965

Upvotes: 1

Related Questions