Delete a folder from Amazon S3 using API

Question

I am trying to delete all the files inside a folder which is basically the date.

Suppose, if there are 100 files under folder "08-10-2015", instead of sending all those 100 file names, i want to send the folder name.

I am trying below code and it is not working for me.

        DeleteObjectsRequest multiObjectDeleteRequest = new DeleteObjectsRequest();

        multiObjectDeleteRequest.BucketName = bucketName;

        multiObjectDeleteRequest.AddKey(keyName + "/" + folderName + "/");


        AmazonS3Config S3Config = new AmazonS3Config()
        {
            ServiceURL = string.Format(servicehost)
        };

        using (IAmazonS3 client = Amazon.AWSClientFactory.CreateAmazonS3Client(accesskey, secretkey, S3Config))
        {
            try
            {
                DeleteObjectsResponse response = client.DeleteObjects(multiObjectDeleteRequest);
                Console.WriteLine("Successfully deleted all the {0} items", response.DeletedObjects.Count);

            }
            catch (DeleteObjectsException e)
            {
                // Process exception.
            }

I am using the above code and it is not working.

Answer 1

Based on Chadwick's answer here is an updated version of the old S3DirectoryInfo.Delete. Now Async, using list V2 and wrapped into a routine for you.

Note it is not well-tested, esp. on >1000 items. Let me know if you battle-test it.

General algorithm

1. List up to 1000 items in folder
2. Iterate and add these to a deletion request
3. Delete them
4. Goto 1 until the list is no longer truncated.

Code:

async void DeleteS3Folder(AmazonS3Client s3Client, string bucketName, string directoryPrefix){
    // Recursively delete all items in a S3 folder. Based on S3DirectoryInfo.Delete
    var listObjectsRequest = new ListObjectsV2Request
    {
        BucketName = bucketName,
        Prefix = directoryPrefix
    };
    
    var deleteObjectsRequest = new DeleteObjectsRequest
    {
        BucketName = bucketName
    };
    
    ListObjectsV2Response listObjectsResponse = null;
    do
    {
        listObjectsResponse = await s3Client.ListObjectsV2Async(listObjectsRequest);
        foreach (S3Object item in listObjectsResponse.S3Objects.OrderBy((S3Object x) => x.Key))
        {
            deleteObjectsRequest.AddKey(item.Key);
            if (deleteObjectsRequest.Objects.Count == 1000)
            {
                await s3Client.DeleteObjectsAsync(deleteObjectsRequest);
                deleteObjectsRequest.Objects.Clear();
            }

            listObjectsRequest.StartAfter = item.Key;
        }
    }
    while (listObjectsResponse.IsTruncated);
    if (deleteObjectsRequest.Objects.Count > 0)
    {
        await s3Client.DeleteObjectsAsync(deleteObjectsRequest);
    }
}

Answer 2

I'm not sure why they didn't keep this method in future SDKs, but, for those interested, here is the implementation of the S3DirectoryInfo.Delete method:

ListObjectsRequest listObjectsRequest = new ListObjectsRequest
            {
                BucketName = bucket,
                Prefix = directoryPrefix
            };
            
            DeleteObjectsRequest deleteObjectsRequest = new DeleteObjectsRequest
            {
                BucketName = bucket
            };
            
            ListObjectsResponse listObjectsResponse = null;
            do
            {
                listObjectsResponse = s3Client.ListObjects(listObjectsRequest);
                foreach (S3Object item in listObjectsResponse.S3Objects.OrderBy((S3Object x) => x.Key))
                {
                    deleteObjectsRequest.AddKey(item.Key);
                    if (deleteObjectsRequest.Objects.Count == 1000)
                    {
                        s3Client.DeleteObjects(deleteObjectsRequest);
                        deleteObjectsRequest.Objects.Clear();
                    }

                    listObjectsRequest.Marker = item.Key;
                }
            }
            while (listObjectsResponse.IsTruncated);
            if (deleteObjectsRequest.Objects.Count > 0)
            {
                s3Client.DeleteObjects(deleteObjectsRequest);
            }

Answer 3

You have to:

1. List all objects in the folder
2. Retrieve key for each object
3. Add this key to a multiple Delete Object Request

4. Make the request to delete all objects

    AmazonS3Config S3Config = new AmazonS3Config()
       {
           ServiceURL = "s3.amazonaws.com",
           CommunicationProtocol = Amazon.S3.Model.Protocol.HTTP,
       };
   
    const string AWS_ACCESS_KEY = "xxxxxxxxxxxxxxxx";
    const string AWS_SECRET_KEY = "yyyyyyyyyyyyyyyy";            
    AmazonS3Client client = new AmazonS3Client(AWS_ACCESS_KEY, AWS_SECRET_KEY, S3Config);
   
       DeleteObjectsRequest request2 = new DeleteObjectsRequest();
       ListObjectsRequest request = new ListObjectsRequest
       {
           BucketName = "yourbucketname",
           Prefix = "yourprefix"
   
       };
   
       ListObjectsResponse response = await client.ListObjectsAsync(request);
               // Process response.
               foreach (S3Object entry in response.S3Objects)
               {
   
                   request2.AddKey(entry.Key);
               }
       request2.BucketName = "yourbucketname";
       DeleteObjectsResponse response2 = await client.DeleteObjectsAsync(request2);
   

Answer 4

I think you can delete the entire folder using the following code:

 AmazonS3Config cfg = new AmazonS3Config();
 cfg.RegionEndpoint = Amazon.RegionEndpoint.EUCentral1;
 string bucketName = "your bucket name";
 AmazonS3Client s3Client = new AmazonS3Client("your access key", "your secret key", cfg);
 S3DirectoryInfo directoryToDelete = new S3DirectoryInfo(s3Client, bucketName, "your folder name or full folder key");
 directoryToDelete.Delete(true); // true will delete recursively in folder inside

I am using amazon AWSSDK.Core and AWSSDK.S3 version 3.1.0.0 for .net 3.5. I hope it can help you