Downloading a file from AWS S3 bucket via AWS SDK corrupts the file

Question

I'm downloading a file (zip) from a bucket in my aws s3 using a java code & aws sdk. However, the downloaded file is corrupted. Downloading the file manually works.

I compared the content of the files and notices that the corrupted file contains kind of uncoded chars see this:

The code that is used to download is the following:

public boolean downloadFile(String bucketName,  String fileNameOnServer, String localFileName )
    {
        S3Object object =null;
        InputStream objectData =null;
        try
        {
            object = s3.getObject(new GetObjectRequest(bucketName, fileNameOnServer));
            objectData = object.getObjectContent();
        }
        catch(Exception e)
        {
            LoggingService.writeToLog("Error001 downloading file "+bucketName+"/"+fileNameOnServer+" to "+localFileName, e, LogModule.CommonUtils,LogLevel.Error);
            return false;
        }
        try
        {
            FileWriter fw = new FileWriter(localFileName, true);
            BufferedWriter bw = new BufferedWriter(fw);
            try
            {
                BufferedReader reader = new BufferedReader(new InputStreamReader(object.getObjectContent()));
                String line;                                                
                while( (line = reader.readLine() ) !=null)
                {
                    bw.write(line);
                    bw.newLine();
                }
                LoggingService.writeToLog("file from "+bucketName+"/"+fileNameOnServer+" "+" downloaded to "+bucketName + " successfully",  LogModule.CommonUtils,LogLevel.Info);
                return true;
            }
            catch(IOException e)
            {
                LoggingService.writeToLog("Error downloading file "+bucketName+"/"+fileNameOnServer+" to "+localFileName, e, LogModule.CommonUtils,LogLevel.Error);
                return false;
            }
            finally
            {
                objectData.close();
                bw.close();
                object.close();
            }

        }
        catch(IOException e)
        {
            LoggingService.writeToLog("Error opening local file "+localFileName+" for writing ", e, LogModule.CommonUtils,LogLevel.Error);
            return false;
        }
    }

Answer 1

The Photo Spring BOOT app now supports downloading an image. It works perfectly. You should use the AWS SDK for Java V2. Here is the example app:

When i open the downloaded image - it is valid and not corrupt:

The code to download this image from an Amazon S3 bucket is located in a Spring Controller:

  @RequestMapping(value = "/downloadphoto", method = RequestMethod.GET)
    void buildDynamicReportDownload(HttpServletRequest request, HttpServletResponse response) {
        try {

            //Get the photo object name
            String photoKey = request.getParameter("photoKey");
            byte[] photoBytes = s3Client.getObjectBytes("myphotobucket", photoKey) ;
            InputStream is = new ByteArrayInputStream(photoBytes);

            //define the required information to download the image
            response.setContentType("image/png");
            response.setHeader("Content-disposition", "attachment; filename="+photoKey);
            org.apache.commons.io.IOUtils.copy(is, response.getOutputStream());
            response.flushBuffer();

        } catch (Exception e) {
            e.printStackTrace();
        }
    }

The V2 S3 code is here. Notice it reads an object and returns a byte[].

public byte[] getObjectBytes (String bucketName, String keyName) {

        s3 = getClient();

        try {
            // create a GetObjectRequest instance
            GetObjectRequest objectRequest = GetObjectRequest
                    .builder()
                    .key(keyName)
                    .bucket(bucketName)
                    .build();

            // get the byte[] from this AWS S3 object
            ResponseBytes<GetObjectResponse> objectBytes = s3.getObjectAsBytes(objectRequest);
            byte[] data = objectBytes.asByteArray();
            return data;

        } catch (S3Exception e) {
            System.err.println(e.awsErrorDetails().errorMessage());
            System.exit(1);
        }
        return null;
    }

Finally the Javascript call

function DownloadImage(){

    //Post the values to the controller
    var photo =  $('#photo').val();
    window.location="../downloadphoto?photoKey=" + photo ;
}

This example will be updated so all of this code is in the example doc.

https://github.com/awsdocs/aws-doc-sdk-examples/tree/master/javav2/usecases/creating_photo_analyzer_app

Answer 2

The documentation for InputStreamReader includes this:

An InputStreamReader is a bridge from byte streams to character streams: It reads bytes and decodes them into characters using a specified charset. The charset that it uses may be specified by name or may be given explicitly, or the platform's default charset may be accepted.

In other words, it attempts to treat the data as text data. For binary data like in a zip file, this will almost assuredly corrupt the data. Instead, if you read and write the bytes directly, you will pass them along without changing them:

package com.exampleapp.app;

import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.GetObjectRequest;
import com.amazonaws.services.s3.model.S3Object;
import com.amazonaws.services.s3.model.S3ObjectInputStream;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.InputStream;
import java.io.IOException;
import java.io.OutputStream;

public class App
{
    public static void main(String[] args)
    {
        downloadFile("example-bucket", "example-key.zip", "local-file.zip");
    }

    static AmazonS3 s3 = AmazonS3ClientBuilder.standard().withRegion(Regions.DEFAULT_REGION).build();

    public static boolean downloadFile(String bucketName,  String fileNameOnServer, String localFileName )
    {
        S3Object object = null;
        InputStream objectData = null;
        InputStream reader = null;
        OutputStream writer = null;
        try
        {
            object = s3.getObject(new GetObjectRequest(bucketName, fileNameOnServer));
            objectData = object.getObjectContent();
        }
        catch(Exception e)
        {
            System.out.println("Error001 downloading file "+bucketName+"/"+fileNameOnServer+" to "+localFileName);
            return false;
        }
        try
        {
            File file = new File(localFileName);
            try
            {
                reader = new BufferedInputStream(object.getObjectContent());
                writer = new BufferedOutputStream(new FileOutputStream(file));
                int read = -1;
                while ( ( read = reader.read() ) != -1 )
                {
                    writer.write(read);
                }
                System.out.println("file from "+bucketName+"/"+fileNameOnServer+" "+" downloaded to "+bucketName + " successfully");
                return true;
            }
            catch(IOException e)
            {
                System.out.println("Error downloading file "+bucketName+"/"+fileNameOnServer+" to "+localFileName);
                return false;
            }
            finally
            {
                object.close();
                writer.flush();
                writer.close();
                reader.close();
            }

        }
        catch(IOException e)
        {
            System.out.println("Error opening local file "+localFileName+" for writing ");
            return false;
        }
    }
}