CODEWITHSUNDEEP
javabase64gzip
ken
ken

Reputation: 9013

Uncompressing base64 gzip on S3 and saving to local file system

I am a Java bonehead/newbie so please be gentle. I have two functions which I realize are somewhat incompatible:

  1. saveS3toFilesystem - takes a InputStream from AWS S3 and saves it to the local filesystem as a file
  2. decompress - takes a string and decodes the base64 encoding and the decompresses the gzip compression.

I really want these two to work in concert to achieve the result of the file saved to the filesystem being the uncompressed file but I realize that my "decompress" function should probably be changed to receive a stream rather than a string but sadly I'm just a "cutter and paster" in the world of Java these days.

Here are my two functions as they are now:

private void saveS3toFilesystem(String filename, String bucketName, String localFilename) {
  S3Object obj = s3.getObject(bucketName, filename);
  InputStream in = obj.getObjectContent();
  try {
    Files.createDirectories(Paths.get(localFilename.replace(this.FILE_EXTENSION, "")));
    Files.copy(in, Paths.get(localFilename));
    this.logger.log("Input file has been placed in local filesystem for ITMS to pick up: " + localFilename + "\n");
  } catch (IOException err) {
    this.logger.log("There was a problem saving the file to " + localFilename);
    err.printStackTrace();
  } finally {
    try {
      in.close();
    } catch (IOException err) {
      err.printStackTrace();
    }
  }
  return;
}

and then ...

private String decompress(String compressedZip) {
  byte[] decodedBytes = Base64.getDecoder().decode(compressedZip);
  String result = null;
  GZIPInputStream zip = null;
  try {
    zip = new GZIPInputStream(new ByteArrayInputStream(decodedBytes));
    result = IOUtils.toString(zip);
  } catch (IOException e) {
    e.printStackTrace();
  } finally {
    IOUtils.closeQuietly(zip);
  }
  return result;
}

Can anyone please help me to achieve the dream? Happy to do it with streams, strings, or any method that will work. Sadly I can't afford atm to up my Java skills enough to grok the solution myself.

Many thanks in advance.

Upvotes: 1

Views: 1268

Answers (1)

Rann Lifshitz
Rann Lifshitz

Reputation: 4090

Based on the following APIs : Base64.Decoder and GZIPInputStream (look at the wrap method on the former and the constructors on the latter), the decompress method can be overloaded as follows: 

private String decompress(InputStream compressedStream) {
  InputStream decodingStream = Base64.getDecoder().wrap(compressedStream);
  String result = null;
  GZIPInputStream zip = null;
  try {
    zip = new GZIPInputStream(decodingStream);
    result = IOUtils.toString(zip);
  } catch (IOException e) {
    e.printStackTrace();
  } finally {
    IOUtils.closeQuietly(zip);
  }
  return result;
}

And finally, the changes to saveS3toFilesystem are as follows : 

private void saveS3toFilesystem(String filename, String bucketName, String localFilename) {
  S3Object obj = s3.getObject(bucketName, filename);
  InputStream in = obj.getObjectContent();
  // decoding the inputstream via decode into a string, which is then
  // used in order to create an inputstream of decoded data
  InputStream decodedStream = 
     new ByteArrayInputStream(decompress(in).getBytes(StandardCharsets.UTF_8));
  try {
    Files.createDirectories(Paths.get(localFilename.replace(this.FILE_EXTENSION, "")));
    Files.copy(decodedStream, Paths.get(localFilename));
    this.logger.log("Input file has been placed in local filesystem for ITMS to pick up: " + localFilename + "\n");
  } catch (IOException err) {
    this.logger.log("There was a problem saving the file to " + localFilename);
    err.printStackTrace();
  } finally {
    try {
      in.close();
    } catch (IOException err) {
      err.printStackTrace();
    }
  }
  return;
}

Upvotes: 1

Related Questions