CODEWITHSUNDEEP
javaamazon-s3
ahmetcetin
ahmetcetin

Reputation: 2990

How to download a file from url without saving it to disk?

I am trying to download files from urls and upload them to S3 buckets in AWS. My best approach so far is saving the file to the temp folder in the disk and delete it after uploading it to S3 bucket.

This is how I do it:

private File downloadFileFromUrl(String fileUrlToUpload) {
    try {
        URL url = new URL(fileUrlToUpload);
        String tempDir = System.getProperty("java.io.tmpdir");
        File file = new File(tempDir + FilenameUtils.getName(url.getPath()));
        FileUtils.copyURLToFile(url, file);
        return file;
    } catch (IOException e) {
        LOGGER.error("Could not download file from: {}!", fileUrlToUpload);
        return null;
    }
}

And then delete the file after uploading it in the service which calls downloadFileFromUrl method.

However, I am looking for a way to download a file from url and upload it directly to S3 bucket without saving it to the disk. I am not even sure if this is a better approach, but it seems cleaner solution than saving something to the disk.

I am also open to other approaches. Thanks!

Upvotes: 2

Views: 2384

Answers (1)

Vüsal
Vüsal

Reputation: 2706

Probably, in this case, the best approach is to use AWS Signed URL to upload and download the file - instead of serving as a proxy.

If you insist on downloading the file yourself, you can do the following: 

BufferedInputStream bufferedInputStream = new BufferedInputStream(url.openStream());
StringBuilder sb = new StringBuilder();
byte[] buffer = new byte[1024];
while ((buffer = bufferedInputStream.read(buffer)) != -1) {
    sb.append(new String(buffer));
}

sb - contains contents of the file.

You can also use Apache Commons Lang: 

IOUtils.toString(new InputStreamReader(url.openStream()));

Upvotes: 3

Related Questions