CODEWITHSUNDEEP
amazon-web-servicesapache-camelamazon-dynamodb
Yuriy Kovalek
Yuriy Kovalek

Reputation: 694

Apache Camel with AWS DynamoDB (aws-ddb) URI parameters

I'm trying to route messages from AWS SQS to AWS Dynamo DB using Apache Camel using the following route definition:

<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
    <route>
        <from uri="aws-sqs://my_queue?accessKey=${aKey}&amp;secretKey=RAW(${sKey})&amp;maxMessagesPerPoll=10&amp;deleteAfterRead=true"/>
        <unmarshal>
            <camel:json library="Jackson"/>
        </unmarshal>

        <to uri="aws-ddb:my_table?accessKey=${aKey}&amp;secretKey=RAW(${sKey})&amp;readCapacity=15&amp;writeCapacity=100&amp;operation=PutItem"/>

    </route>
</camelContext>

But on execution Camel complains that the Dynamo Db URI is missing some required parameters:

org.apache.camel.ResolveEndpointFailedException: Failed to resolve endpoint: aws-ddb://table?amazonDDBClient=%23ddbClient&amazonDdbEndpoint=ap-southeast-2&readCapacity=10&writeCapacity=10 due to: Status Code: 400, AWS Service: AmazonDynamoDB, AWS Request ID: KHA79STK78SHC2BG2R8HLPF7RJVV4KQNSO5AEMVJF66Q9ASUAAJG, AWS Error Code: ValidationException, AWS Error Message: 2 validation errors detected: Value null at 'keySchema.hashKeyElement.attributeName' failed to satisfy constraint: Member must not be null; Value null at 'keySchema.hashKeyElement.attributeType' failed to satisfy constraint: Member must not be null

The fun part is, these 2 parameters are not described anywhere in Camel DDB doc. I spent some time browsing the Camel source and found 2 undocumented URI parameters: keyAttributeName & keyAttributeType, which worked perfectly for me. (I hope this find of mine helps someone as well).

Now even more fun is that these should not be requested for inserting items into Dynamo DB, but I can no longer reproduce this error when I remove the 2 undocumented params from the URI.

So my questions are:

  1. Why did AWS request hash key data for PutItem requests?
  2. Why can't I reproduce this behaviour any longer?

I could not find any hints either in Camel or AWS documentation, googling only uncovers a handful of irrelevant results.

Upvotes: 2

Views: 1216

Answers (1)

ChillyPro
ChillyPro

Reputation: 182

Could it be that you didn't specify the region your Dynamo DB table belongs too?

I had the same problem and found that my data was stored in the US East region instead of the region I had my table in. And to answer your two questions then it would be:

  1. Because the table didn't exist, Camel AWS tries to create it for you as can be seen in the DdbEndpoint class. Personally I don't like this side effect but it is there :/ This means that those attributes suddenly got required.

  2. You cannot reproduce it after the first try because now it is created :) Delete the table and you will be able to reproduce it.

So my solution was to specify my own AmazonDynamoDBClient with the region set and then put it in the registry so it could be found from the route with the parameter: amazonDDBClient

Hope it helps!

Upvotes: 1

Related Questions