CODEWITHSUNDEEP
aws-glueaws-glue-data-catalogglue-crawler
Nikhil C
Nikhil C

Reputation: 11

Glue crawler could not classify JSON data of size > 20 mb

I am a newbie to AWS Glue. I am having a raw data in s3 in JSON format with size > 20 MB. After creating a crawler over this JSON data, classification and compression type are showing as 'UNKNOWN' when look into the table details.

I have tried creating a custom classifier with jsonPath $[*]. Still I am not able get the classification correctly.

Could anyone please help me with this?

Upvotes: 0

Views: 292

Answers (1)

Afzal Abdul Azeez
Afzal Abdul Azeez

Reputation: 305

Create a custom classifier and add jsonPath with the array name in the json object and add the classifier in the corresponding crawler

Eg : $arrayname[*]

Upvotes: 0

Related Questions