CODEWITHSUNDEEP
javagoogle-cloud-storage
Karthic Rao
Karthic Rao

Reputation: 3754

Google Cloud Storage Java XML API chunk by chunk file upload

I'm using Google Cloud Storage Java XML API to upload a file from client into the Google Cloud Storage bucket. The file is getting uploaded successfully but I'm reading the entire file into the buffer before uploading it, thus I'm forced to allocate a buffer which is as big as size of the file.

How to do a chunk by chunk upload so that I don't need to read entire file content into the buffer at once? Following is the piece of code I've used. What are the modifications to be done on this so that I could do a chunk by chunk upload?

httpTransport = GoogleNetHttpTransport.newTrustedTransport();


String p12Content = Files.readFirstLine(new File("key.p12"), Charset.defaultCharset());
            Preconditions.checkArgument(!p12Content.startsWith("Please"), p12Content);
            //[START snippet]
            // Build a service account credential.
Path path = Paths.get(MyFilePath);
    byte[] byteArray = java.nio.file.Files.readAllBytes(path);
HttpContent contentsend = new ByteArrayContent("text/plain", byteArray );

GoogleCredential credential = new GoogleCredential.Builder().setTransport(httpTransport)
                .setJsonFactory(JSON_FACTORY)
                .setServiceAccountId(SERVICE_ACCOUNT_EMAIL)
                .setServiceAccountScopes(Collections.singleton(STORAGE_SCOPE))
                .setServiceAccountPrivateKeyFromP12File(new File("key.p12"))

                .build();
            // Set up and execute a Google Cloud Storage request.
String URI = "https://storage.googleapis.com/" + BUCKET_NAME + "/myfile";
HttpRequestFactory requestFactory = httpTransport.createRequestFactory(credential);
GenericUrl url = new GenericUrl(URI);
HttpRequest request = requestFactory.buildPutRequest(url,contentsend);
HttpResponse response = request.execute();

Upvotes: 3

Views: 528

Answers (1)

PrecariousJimi
PrecariousJimi

Reputation: 1543

With XML API you can either implement resumable uploads (sorry, didn't find any code samples) or use object composition, which might be easier to start with:

  • divide your data locally into as many chunks as required
  • upload each chunk as a distinct object
  • compose your final object
  • delete any temporary objects

Upvotes: 1

Related Questions