CODEWITHSUNDEEP
sdkboto3aws-gluedelta-lakeglue-crawler
Saad Ather
Saad Ather

Reputation: 21

Unknown parameter in Targets DeltaTargets must be one of: S3Targets, JdbcTargets, MongoDBTargets, DynamoDBTargets, CatalogTargets

Ok so i am trying to create a glue crawler that will crawler over the delta path and create a delta table, in the official AWS SDK documentation the syntax is written and I'm following as it is AWS SDK Documentation

Here is my code:

import boto3

bucket_name = "data-extract"
table_path = "BatchTable"

s3_target = f"s3://{bucket_name}/datalake/{table_path}"
glue_client = boto3.client('glue')

response = glue_client.create_crawler(
    Name=f"{bucket_name}-{table_path}-crawler_test",
    Role='AWSGlueServiceRoleDefault',
    DatabaseName='crispy',
    Targets={"DeltaTargets": [{'DeltaTables': "s3://data-extract/datalake/BatchTable/",'CreateNativeDeltaTable': True}]})

Upvotes: 1

Views: 104

Answers (1)

Saad Ather
Saad Ather

Reputation: 21

OK so after so much look around i found out the boto3 version was the culprit, normally in Glue 3.0 its using the boto3 version of 1.18.10 and in this version the "DeltaTargets" are not supported, so to use this we need at least a boto3 version of > 1.24.0. Also to completely use all the delta features use the boto3 version of 1.28.4. Changing the job to Glue 4.0 successfully created a crawler for delta table.

Upvotes: 1

Related Questions