CODEWITHSUNDEEP
google-cloud-storage
user2755358
user2755358

Reputation: 119

Timeout in uploading a big file to google cloud storage

I'm having trouble uploading large files to Google Cloud Storage. I successfully uploaded a 700MB file, but when I tried a 5GB text file, it threw the following exception. I was unable to find a solution with a Google search.

The problem is in the main method of a simple java class.

Exception in thread "main" java.lang.RuntimeException: java.net.SocketTimeoutException: Read timed out

Caused by: java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(SocketInputStream.java:150)
    at java.net.SocketInputStream.read(SocketInputStream.java:121)
    at sun.security.ssl.InputRecord.readFully(InputRecord.java:312)
    at sun.security.ssl.InputRecord.read(InputRecord.java:350)
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:893)
    at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:850)
......

Upvotes: 4

Views: 6448

Answers (3)

SANN3
SANN3

Reputation: 10069

In Java SDK we have option to change the connection timeout and retry.

HttpTransportOptions transportOptions = StorageOptions.getDefaultHttpTransportOptions();
            transportOptions = transportOptions.toBuilder().setConnectTimeout(60000).setReadTimeout(60000)
                    .build();
var storage = StorageOptions.newBuilder()
    .setRetrySettings(RetrySettings.newBuilder().setMaxAttempts(2).build())
    .setTransportOptions(transportOptions)
    .setProjectId("project_id").build().getService();

Upvotes: 1

Benson
Benson

Reputation: 22847

With larger files, especially on mobile devices and wireless connections, you're much more likely to have your uploads interrupted by a broken connection. The solution to this is to make your upload resilient against broken connections. Google Cloud Storage handles this using a technique called Resumable Uploads. You'll need to make use of this technique so your application can recover from network issues.

Upvotes: 1

user2366842
user2366842

Reputation: 1216

Getting java.net.SocketTimeoutException: Connection timed out in android it looks like you may need to jump up your connection timeout setting. The link is for android, but the same thing applies, and it's implemented exactly the same.

Upvotes: 1

Related Questions