CODEWITHSUNDEEP
amazon-web-servicesamazon-dynamodbaws-glue
justHelloWorld
justHelloWorld

Reputation: 6828

Can AWS Glue write to DynamoDB?

I need to do some grouping job from a Source DynamoDB table, then write each resulting Item to another Target DynamoDB table (or a secondary index of the Source one).

Here I see that DynamoDB can be used as a Source (as well as reported in Connection Types).

However, it's not clear to me if a DynamoDB table can be used as Target as well.

Note: each resulting grouping item must be written into a separate DynamoDB Item (i.e., if there are X objects resulting from grouping, X Items must be written to Target DynamoDB table).

Upvotes: 7

Views: 13081

Answers (2)

Lydon
Lydon

Reputation: 329

Glue can now read and write to DynamoDB. The option to write is not available via the console, but can be done by editing the script. Example:

Datasink1 = glueContext.write_dynamic_frame.from_options(
    frame=ApplyMapping_Frame1,
    connection_type="dynamodb",
    connection_options={
        "dynamodb.output.tableName": "myDDBTable",
        "dynamodb.throughput.write.percent": "1.0"
    }
)

As per:

Upvotes: 11

rhlarora84
rhlarora84

Reputation: 77

The Glue Job scripts can be customized to write to any datasource. If you are using the auto generated scripts, you can add boto3 library to write to DynamoDb tables.

If you want to test the scripts easily, you can create a Dev endpoint through AWS console & launch a jupyter notebook to write and test your glue job scripts.

Upvotes: 3

Related Questions