CODEWITHSUNDEEP
filepicker.io
Ben
Ben

Reputation: 1302

Multiple s3 buckets in Filepicker.io

I need to upload to multiple s3 buckets with filepicker.io. I found a tweet that indicated that there was a hacky, but possible, way to do this. Support hasn't gotten back to me yet, so I'm hoping that someone here already knows the answer!

Upvotes: 1

Views: 454

Answers (3)

Annabel
Annabel

Reputation: 1414

@Ben

I am developing code with same issue of files needing to go into many buckets. I'm working in ASP.net.

What I have done is have one Filepicker 'application' with it's own S3 bucket. I already had a callback to the server in the javascript onSuccess() function (which is passed as a parameter to filepicker.store()). This callback needed to be there to do some book-keeping anyway.

So I have just added in an extra bit to the server-side callback code which uses the AWS SDK to copy the object from the bucket filepicker uploades it to, to it's final destination bucket.

This is my C# code for moving, or rather copying, an object between buckets:

public bool MoveObject(string bucket1, string key1, string bucket2, string key2 = null)
{
    bool success = false;
    if (key2 == null) key2 = key1;
    Logger logger = new Logger(); // my logging system
    try
    {
        RegionEndpoint region = RegionEndpoint.EUWest1; // use your region here
        using (AmazonS3Client s3Client = new AmazonS3Client(region))
        {
            // TODO: CheckForBucketFunction
            CopyObjectRequest request = new CopyObjectRequest();
            request.SourceBucket = bucket1;
            request.SourceKey = key1;
            request.DestinationBucket = bucket2;
            request.DestinationKey = key2;

            S3Response response = s3Client.CopyObject(request);
            logger.Info2Log("response xml = \n{0}\n", response.ResponseXml);
            response.Dispose(); 
            success = true;
        }
    }
    catch (AmazonS3Exception ex)
    {
        logger.Info2Log("Error copying file between buckets: {0} - {1}", 
            ex.ErrorCode, ex.Message);
        success = false;
    }

    return success;
}

There are AWS SDKs for other server languages and the good news is Amazon doesn't charge for copying objects between buckets in the same region.

Now I just have to decide how to delete the object from the filepicker application bucket. I could do it on the server using more AWS SDK code but that will be messy as it leaves links to the object in the filepicker console. Or I could do it from the browser using filepicker code.

Upvotes: 1

Ben
Ben

Reputation: 1302

Support just got back to me. There's no way to do this besides creating multiple applications, which is okay if you are just switching between prod/staging/dev, but not a good solution if you have to upload to arbitrary buckets.

My solution is to execute a PUT request with the x-amz-copy-source header after the file has been uploaded, which copies it to the correct bucket.

This is pretty hacky as it request two extra requests per file -- one filepicker.stat and one more call to s3 (or your server).

Upvotes: 1

ezkl
ezkl

Reputation: 3861

Have you tried generating a second application/API key? It looks like they lock your S3/AWS credentials to an application/API key rather than directly to the account.

Upvotes: 2

Related Questions