Using Java AWS S3 1.9.8 to access StorageGRID Webscale

Question

I'm trying to connect to our StorageGRID Webscale system with java using aws-java-sdk and am getting the following error:

Exception in thread "main" com.amazonaws.AmazonClientException: Failed to parse XML document with handler class com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser$ListAllMyBucketsHandler
at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:128)
at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseListMyBucketsResponse(XmlResponsesSaxParser.java:320)
at com.amazonaws.services.s3.model.transform.Unmarshallers$ListBucketsOwnerUnmarshaller.unmarshall(Unmarshallers.java:50)
at com.amazonaws.services.s3.model.transform.Unmarshallers$ListBucketsOwnerUnmarshaller.unmarshall(Unmarshallers.java:46)
at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:62)
at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:31)
at com.amazonaws.http.response.AwsResponseHandlerAdapter.handle(AwsResponseHandlerAdapter.java:70)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleResponse(AmazonHttpClient.java:1428)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1135)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:948)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:661)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:635)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:618)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$300(AmazonHttpClient.java:586)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:573)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:445)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4039)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3976)
at com.amazonaws.services.s3.AmazonS3Client.getS3AccountOwner(AmazonS3Client.java:859)
at com.amazonaws.services.s3.AmazonS3Client.getS3AccountOwner(AmazonS3Client.java:851)
at com.graybar.awstest.Controller.main(Controller.java:37)
Caused by: org.xml.sax.SAXParseException; lineNumber: 48; columnNumber: 3; The element type "link" must be terminated by the matching end-tag "</link>".
at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:203)
at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(ErrorHandlerWrapper.java:177)
at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:441)
at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:368)
at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(XMLScanner.java:1436)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1749)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2973)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:606)
at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:117)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:510)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:848)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:777)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseXmlInputStream(XmlResponsesSaxParser.java:114)
... 20 more

My maven dependency for aws looks like this:

    <dependency>
        <groupId>com.amazonaws</groupId>
        <artifactId>aws-java-sdk</artifactId>
        <version>1.9.8</version>
    </dependency>

The code that I'm using is this:

import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.services.s3.AmazonS3Client;
import com.amazonaws.services.s3.S3ClientOptions;
import com.amazonaws.services.s3.model.Bucket;
import com.amazonaws.services.s3.model.Owner;

public class Controller {
public static void main(String[] args) {
    final String profile = "default";
    final String address = "https://my_ip_address:443";

    // use path-style access to avoid issues with DNS-incompatible bucket names
    final S3ClientOptions options = new S3ClientOptions();
    options.setPathStyleAccess(true);

    // will load credentials from ~/.aws/credentials
    final AmazonS3Client s3 = new AmazonS3Client(new ProfileCredentialsProvider(profile));
    s3.setEndpoint(address);
    s3.setS3ClientOptions(options);

    Owner owner = s3.getS3AccountOwner();
    System.out.println("Owner: " + owner.getId());
    for (Bucket b : s3.listBuckets()) {
        System.out.println("Bucket: " + b);
    }
}
}

I am following the instructions given in this blog:

https://community.netapp.com/t5/Technology/Accessing-StorageGRID-Webscale-through-its-S3-API/ba-p/98225

I cannot seem to get past this error. Any ideas?

Answer 1

The problem here was that I was pointing my code to the wrong server and port. I changed my 'address' variable to this:

final String address = "https://mystoragegridserver:8082";

We also had to get a certificate loaded onto the server to handle SSL.