CODEWITHSUNDEEP
amazon-web-servicesaws-sdkaws-glueaws-glue-data-catalog
Ryan
Ryan

Reputation: 11

AWS Glue - boto3 crawler not creating table

I am trying to create and run a AWS glue crawler through the boto3 library. The crawler is going against JSON files in an s3 folder. The crawler completes successfully, when i check the logs there are no errors but it doesn't create any table in my glue database

It's not a permission issue as I am able to create the same crawler through a CFT and when I run that it creates the table as expected. Im using the same role as my CFT in my code I'm running with boto3 to create it.

Have tried using boto3 create_crawler() and run_crawler(). Tried using boto3 update_crawler() on the crawler created from the CFT and updating the s3 target path. 

 response = glue.create_crawler(
        Name='my-crawler',
        Role='my-role-arn',
        DatabaseName='glue_database',
        Description='Crawler for generating table from s3 target',
        Targets={
            'S3Targets': [
                {
                    'Path': s3_target
                }
            ]
        },
        SchemaChangePolicy={
            'UpdateBehavior': 'UPDATE_IN_DATABASE',
            'DeleteBehavior': 'LOG'
        },
        TablePrefix=''
    )

Upvotes: 0

Views: 1590

Answers (1)

Sandeep Fatangare
Sandeep Fatangare

Reputation: 2144

Are you sure you have passed correct region in glue client (glue object creation).

Once I copied code and forgot to change region and spent hours figuring out why it is not creating table when there is no error. Eventually I ended in figuring out that table is created in another region as I forgot to change region while I copied code to new region.

Upvotes: 0

Related Questions