CODEWITHSUNDEEP
google-cloud-platformgoogle-cloud-storage
Andreas Lundgren
Andreas Lundgren

Reputation: 12545

How to upload to Google Storage using JAVA library with WriteChannel and MD5 check

How can I perform an MD5 check of uploaded content when using an output stream that is written to a writer channel? When I do Storage::create, the MD5 in the provided BlobInfo is overwritten with the EMPTY_BYTE_ARRAY_MD5. This makes sense since when the blob is created at first, is indeed empty. But I would expect some method on the writer to set an updated MD5 that is expected to be valid once the writer is closed. I cannot find a way to accomplish this, is it possible? I'll attach my code:

Gradle:

api 'com.google.cloud:google-cloud-storage:1.51.0'

Java code:

import com.google.cloud.WriteChannel;
import com.google.cloud.storage.*;
import com.google.common.io.ByteStreams;

import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.channels.Channels;
import java.nio.file.Path;

class StorageServiceImpl {

    @Inject
    private Storage storage;

    public BlobInfo uploadFile(final Path localFile, final String bucketName, final String fileName, final String downloadFileName) throws IOException {
        Blob blob = null;
        String checksum = md5(localFile);
        try (InputStream inputStream = new FileInputStream(localFile.toFile())) {
            blob = storage.create(
                BlobInfo.newBuilder(bucketName, fileName)
                    .setContentType("application/octet-stream")
                    .setContentDisposition(String.format("attachment; filename=\"%s\"", downloadFileName))
                    .setMd5(checksum)
                    .build()
            );
            try (WriteChannel writer = blob.writer(Storage.BlobWriteOption.md5Match())) {
                ByteStreams.copy(inputStream, Channels.newOutputStream(writer));
            }
        } catch (StorageException ex) {
            if (!(400 == ex.getCode() && "invalid".equals(ex.getReason()))) {
                throw ex;
            }
        }
        return blob;
    }
}

This issue appeared once we migrated from the deprecated method Blob create(BlobInfo blobInfo, InputStream content, BlobWriteOption... options);.

Upvotes: 4

Views: 3135

Answers (1)

Andreas Lundgren
Andreas Lundgren

Reputation: 12545

Answer from Google Cloud Support

Don't create a blob and request a writer to the blob. Instead, request a writer to the storage and provide the blob info to that request.

import com.google.cloud.WriteChannel;
import com.google.cloud.storage.*;
import com.google.common.io.ByteStreams;

import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.nio.channels.Channels;
import java.nio.file.Path;

class StorageServiceImpl {

    @Inject
    private Storage storage;

    public BlobInfo uploadFile(final Path localFile, final String bucketName, final String fileName, final String downloadFileName) throws IOException {
        BlobInfo blobInfo = null;
        String checksum = md5(localFile);
        try (InputStream inputStream = new FileInputStream(localFile.toFile())) {
            blobInfo =
                BlobInfo.newBuilder(bucketName, fileName)
                    .setContentType("application/octet-stream")
                    .setContentDisposition(String.format("attachment; filename=\"%s\"", downloadFileName))
                    .setMd5(checksum)
                    .build();
            try (WriteChannel writer = storage.writer(blobInfo, Storage.BlobWriteOption.md5Match())) {
                ByteStreams.copy(inputStream, Channels.newOutputStream(writer));
            }
        } catch (StorageException ex) {
            if (!(400 == ex.getCode() && "invalid".equals(ex.getReason()))) {
                throw ex;
            }
        }
        return blobInfo;
    }
}

Upvotes: 9

Related Questions