CODEWITHSUNDEEP
javaamazon-s3
Digit
Digit

Reputation: 1939

delete list of objects from Amazon s3 in java

Here is my code:

try {
  DeleteObjectsResult delObjRes = s3Client
      .deleteObjects(multiObjectDeleteRequest);
  System.out.format("Successfully deleted all the %s items.\n", delObjRes
      .getDeletedObjects().size());

} catch (MultiObjectDeleteException e) {
  System.out.format("%s \n", e.getMessage());
  System.out.format("No. of objects successfully deleted = %s\n", e
      .getDeletedObjects().size());
  System.out.format("No. of objects failed to delete = %s\n", e.getErrors()
      .size());
  System.out.format("Printing error data...\n");
  for (DeleteError deleteError : e.getErrors()) {
    System.out.format("Object Key: %s\t%s\t%s\n", deleteError.getKey(),
        deleteError.getCode(), deleteError.getMessage());
  }
}

exception is as follows:

Exception in thread "main" Status Code: 400, AWS Service: Amazon S3, AWS Request ID: 3EB96BFE84959731, AWS Error Code: MalformedXML, AWS Error Message: The XML you provided was not well-formed or did not validate against our published schema, S3 Extended Request ID: pE+pEHF36KqItpx1y6tJe6m50lTD1C/YHe0bVOmJW5TRBV7EfxvS5+Dc6JKX5AYb
    at 

    com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:556)
        at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:289)
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:170)
        at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:2648)
        at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:2620)
        at com.amazonaws.services.s3.AmazonS3Client.deleteObjects(AmazonS3Client.java:1363)
        at com.neem23.cleanup.CleanUp.deleteMultipleObjects(CleanUp.java:73)
        at com.neem23.cleanup.StartCleanUp.main(StartCleanUp.java:50)

Upvotes: 6

Views: 4156

Answers (3)

Indi
Indi

Reputation: 429

As you want to delete multiple S3 objects in a specific source folder in an s3 bucket following code can be used.

Here first list the objects of the folder and delete the list.

Listing objects

public static void listFiles(S3Request s3Request) {
        S3Validation.validateSource(s3Request);
        ListObjectsRequest listObjectRequest= new ListObjectsRequest().withBucketName(s3Request.getBucketName()).withPrefix(s3Request.getSourcekey());
        ObjectListing objectListing = AWSClient.getS3Client().listObjects(listObjectRequest);
        List<S3ObjectSummary> objectSummary=objectListing.getObjectSummaries();
        response.setList(objectSummary);
        }

Deleting list of objects

        S3Response s3Response = listFiles(s3Request);
        String setDest=s3Request.getDestKey();
        List<S3ObjectSummary> objectSummary=s3Response.getList();
        for (S3ObjectSummary summary : objectSummary) {
                s3Request.setSourcekey(summary.getKey()); 
                s3Request.setDestKey(setDest+FilenameUtils.getName(summary.getKey()));
                DeleteObjectRequest deleteObjectRequest =new DeleteObjectRequest(s3Request.getBucketName(), s3Request.getSourcekey());
                AWSClient.getS3Client().deleteObject(deleteObjectRequest);
            }
    }

Here S3Request class is the pojo class. You can create a unit test with setters. (s3Request.setBucketName("<bucketName>"), s3Request.setSourcekey("<keypath_of_the_folder>")

Upvotes: 0

Kai Inkinen
Kai Inkinen

Reputation: 2631

This question seems to be quite old, but since I happened to run into a similar issue today, it still deserves an answer. Hopefully someone else finds this page with less googling.

It Turns out I hadn't read the docs properly. Have you verified the number of objects you're trying to delete? At least the current docs (as of 2014.11.12) state that the max number of entries to delete in one go is 1000. I checked the older version 1.3.9 of the javadocs, and they don't state this limitation, but the latest ones do. So, make sure you check the number of entires in the request, and split it up into multiple parts, in case there's too many keys.

Upvotes: 3

Joe Bane
Joe Bane

Reputation: 1636

I have seen this service exception when the List<KeyVersion> that I set on the DeleteObjectsRequest object (via the setKeys method) is empty. Looking at the code you posted in the comments above, this may be the cause of the problem you are experiencing:

List<KeyVersion> keys = new ArrayList<KeyVersion>();
for (String keyName : listOfNames) {
  if (keyName != null && !keyName.isEmpty()) keys.add(new KeyVersion(keyName));
}

/* The keys list could be empty here! */

multiObjectDeleteRequest.setKeys(keys);

Upvotes: 1

Related Questions