Reputation: 3245
I would like to check if a key exists in a given bucket using Java. I looked at the API but there aren't any methods that are useful. I tried to use getObject
but it threw an exception.
Upvotes: 141
Views: 240535
Reputation: 5230
There's now a doesObjectExist method in the official Java API.
Enjoy!
EDIT: This works only in AWS SDK for Java 1.x.
Upvotes: 355
Reputation: 651
The right way to do it in SDK V2, without the overload of actually getting the object, is to use S3Client.headObject()
.
Officially backed by AWS Change Log in the [4.1.1. S3 Operation Migration][2] section.
public boolean exists(String bucket, String key) {
try {
HeadObjectResponse headResponse = client
.headObject(HeadObjectRequest.builder().bucket(bucket).key(key).build());
return true;
} catch (NoSuchKeyException e) {
return false;
}
}
More info about the HeadObject operation can be found in the official documentation: HeadObject.
Upvotes: 65
Reputation: 17
Maybe this will work ?
/**
* Exist object
*
* @param bucketName - Bucket name
* @param key - Object key
* @return Mono<Boolean>
*/
public Mono<Boolean> existObject(String bucketName, String key) {
return Mono.fromFuture(client.headObject(HeadObjectRequest.builder().key(key).bucket(bucketName).build()))
.flatMap(result -> Mono.just(true))
.onErrorResume(error -> Mono.just(false));
}
Upvotes: 0
Reputation: 1964
Alternatively you can use Minio-Java client library, its Open Source and compatible with AWS S3 API.
You can use Minio-Java StatObject.java examples for the same.
import io.minio.MinioClient;
import io.minio.errors.MinioException;
import java.io.InputStream;
import java.io.IOException;
import java.security.NoSuchAlgorithmException;
import java.security.InvalidKeyException;
import org.xmlpull.v1.XmlPullParserException;
public class GetObject {
public static void main(String[] args)
throws NoSuchAlgorithmException, IOException, InvalidKeyException, XmlPullParserException, MinioException {
// Note: YOUR-ACCESSKEYID, YOUR-SECRETACCESSKEY and my-bucketname are
// dummy values, please replace them with original values.
// Set s3 endpoint, region is calculated automatically
MinioClient s3Client = new MinioClient("https://s3.amazonaws.com", "YOUR-ACCESSKEYID", "YOUR-SECRETACCESSKEY");
InputStream stream = s3Client.getObject("my-bucketname", "my-objectname");
byte[] buf = new byte[16384];
int bytesRead;
while ((bytesRead = stream.read(buf, 0, buf.length)) >= 0) {
System.out.println(new String(buf, 0, bytesRead));
}
stream.close();
}
}
I hope it helps.
Disclaimer : I work for Minio
Upvotes: -1
Reputation: 2115
As others have mentioned, for the AWS S3 Java SDK 2.10+ you can use the HeadObjectRequest object to check if there is a file in your S3 bucket. This will act like a GET request without actually getting the file.
Example code since others haven't actually added any code above:
public boolean existsOnS3 () throws Exception {
try {
S3Client s3Client = S3Client.builder ().credentialsProvider (...).build ();
HeadObjectRequest headObjectRequest = HeadObjectRequest.builder ().bucket ("my-bucket").key ("key/to/file/house.pdf").build ();
HeadObjectResponse headObjectResponse = s3Client.headObject (headObjectRequest);
return headObjectResponse.sdkHttpResponse ().isSuccessful ();
}
catch (NoSuchKeyException e) {
//Log exception for debugging
return false;
}
}
Upvotes: 4
Reputation: 39
I also faced this problem when I used
String BaseFolder = "3patti_Logs";
S3Object object = s3client.getObject(bucketName, BaseFolder);
I got error key not found
When I hit and try
String BaseFolder = "3patti_Logs";
S3Object object = s3client.getObject(bucketName, BaseFolder+"/");
it worked , this code is working with 1.9 jar otherwise update to 1.11 and use doesObjectExist as said above
Upvotes: 1
Reputation: 83393
In Amazon Java SDK 1.10+, you can use getStatusCode()
to get the status code of the HTTP response, which will be 404 if the object does not exist.
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.model.AmazonS3Exception;
import org.apache.http.HttpStatus;
try {
AmazonS3 s3 = new AmazonS3Client();
ObjectMetadata object = s3.getObjectMetadata("my-bucket", "my-client");
} catch (AmazonS3Exception e) {
if (e.getStatusCode() == HttpStatus.SC_NOT_FOUND) {
// bucket/key does not exist
} else {
throw e;
}
}
getObjectMetadata()
consumes fewer resources, and the response doesn't need to be closed like getObject()
.
In previous versions, you can use getErrorCode()
and check for the appropriate string (depends on the version).
Upvotes: 19
Reputation: 36051
Update:
It seems there's a new API to check just that. See another answer in this page: https://stackoverflow.com/a/36653034/435605
Original post:
Use errorCode.equals("NoSuchKey")
try {
AmazonS3 s3 = new AmazonS3Client(new ClasspathPropertiesFileCredentialsProvider());
String bucketName = getBucketName();
s3.createBucket(bucketName);
S3Object object = s3.getObject(bucketName, getKey());
} catch (AmazonServiceException e) {
String errorCode = e.getErrorCode();
if (!errorCode.equals("NoSuchKey")) {
throw e;
}
Logger.getLogger(getClass()).debug("No such key!!!", e);
}
Note about the exception: I know exceptions should not be used for flow control. The problem is that Amazon didn't provide any api to check this flow - just documentation about the exception.
Upvotes: 68
Reputation: 2220
The other answers are for AWS SDK v1. Here is a method for AWS SDK v2 (currently 2.3.9).
Note that getObjectMetadata
and doesObjectExist
methods are not currently in the v2 SDK! So those are no longer options. We are forced to use either getObject
or listObjects
.
listObjects
calls are currently 12.5 times more expensive to make than getObject
. But AWS also charges for any data downloaded, which raises the price of getObject
if the file exists. As long as the file is very unlikely to exist (for example, you have generated a new UUID key randomly and just need to double-check that it isn't taken) then calling getObject
is significantly cheaper by my calculation.
Just to be on the safe side though, I added a range()
specification to ask AWS to only send a few bytes of the file. As far as I know the SDK will always respect this and not charge you for downloading the whole file. But I haven't verified that so rely on that behavior at your own risk! (Also, I'm not sure what how range
behaves if the S3 object is 0 bytes long.)
private boolean sanityCheckNewS3Key(String bucket, String key) {
ResponseInputStream<GetObjectResponse> resp = null;
try {
resp = s3client.getObject(GetObjectRequest.builder()
.bucket(bucket)
.key(key)
.range("bytes=0-3")
.build());
}
catch (NoSuchKeyException e) {
return false;
}
catch (AwsServiceException se) {
throw se;
}
finally {
if (resp != null) {
try {
resp.close();
} catch (IOException e) {
log.warn("Exception while attempting to close S3 input stream", e);
}
}
}
return true;
}
}
Note: this code assumes s3Client
and log
are declared and initialized elsewhere. Method returns a boolean, but can throw exceptions.
Upvotes: 3
Reputation: 788
Using Object isting. Java function to check if specified key exist in AWS S3.
boolean isExist(String key)
{
ObjectListing objects = amazonS3.listObjects(new ListObjectsRequest().withBucketName(bucketName).withPrefix(key));
for (S3ObjectSummary objectSummary : objects.getObjectSummaries())
{
if (objectSummary.getKey().equals(key))
{
return true;
}
}
return false;
}
Upvotes: 2
Reputation: 908
Break your path into bucket and object.
Testing the bucket using the method doesBucketExist
,
Testing the object using the size of the listing (0 in case not exist).
So this code will do:
String bucket = ...;
String objectInBucket = ...;
AmazonS3 s3 = new AmazonS3Client(...);
return s3.doesBucketExist(bucket)
&& !s3.listObjects(bucket, objectInBucket).getObjectSummaries().isEmpty();
Upvotes: 3
Reputation: 551
There is an easy way to do it using jetS3t API's isObjectInBucket() method.
Sample code:
ProviderCredentials awsCredentials = new AWSCredentials(
awsaccessKey,
awsSecretAcessKey);
// REST implementation of S3Service
RestS3Service restService = new RestS3Service(awsCredentials);
// check whether file exists in bucket
if (restService.isObjectInBucket(bucket, objectKey)) {
//your logic
}
Upvotes: 1
Reputation: 2613
For PHP (I know the question is Java, but Google brought me here), you can use stream wrappers and file_exists
$bucket = "MyBucket";
$key = "MyKey";
$s3 = Aws\S3\S3Client->factory([...]);
$s3->registerStreamWrapper();
$keyExists = file_exists("s3://$bucket/$key");
Upvotes: 5
Reputation: 1390
This java code checks if the key (file) exists in s3 bucket.
public static boolean isExistS3(String accessKey, String secretKey, String bucketName, String file) {
// Amazon-s3 credentials
AWSCredentials myCredentials = new BasicAWSCredentials(accessKey, secretKey);
AmazonS3Client s3Client = new AmazonS3Client(myCredentials);
ObjectListing objects = s3Client.listObjects(new ListObjectsRequest().withBucketName(bucketName).withPrefix(file));
for (S3ObjectSummary objectSummary: objects.getObjectSummaries()) {
if (objectSummary.getKey().equals(file)) {
return true;
}
}
return false;
}
Upvotes: 4
Reputation: 147
Use ListObjectsRequest setting Prefix as your key.
.NET code:
public bool Exists(string key)
{
using (Amazon.S3.AmazonS3Client client = (Amazon.S3.AmazonS3Client)Amazon.AWSClientFactory.CreateAmazonS3Client(m_accessKey, m_accessSecret))
{
ListObjectsRequest request = new ListObjectsRequest();
request.BucketName = m_bucketName;
request.Prefix = key;
using (ListObjectsResponse response = client.ListObjects(request))
{
foreach (S3Object o in response.S3Objects)
{
if( o.Key == key )
return true;
}
return false;
}
}
}.
Upvotes: 5
Reputation: 1267
Using the AWS SDK use the getObjectMetadata method. The method will throw an AmazonServiceException if the key doesn't exist.
private AmazonS3 s3;
...
public boolean exists(String path, String name) {
try {
s3.getObjectMetadata(bucket, getS3Path(path) + name);
} catch(AmazonServiceException e) {
return false;
}
return true;
}
Upvotes: 26
Reputation: 8431
Use the jets3t library. Its a lot more easier and robust than the AWS sdk. Using this library you can call, s3service.getObjectDetails(). This will check and retrieve only the details of the object (not the contents) of the object. It will throw a 404 if the object is missing. So you can catch that exception and deal with it in your app.
But in order for this to work, you will need to have ListBucket access for the user on that bucket. Just GetObject access will not work. The reason being, Amazon will prevent you from checking for the presence of the key if you dont have ListBucket access. Just knowing whether a key is present or not, will also suffice for malicious users in some cases. Hence unless they have ListBucket access they will not be able to do so.
Upvotes: 0