CODEWITHSUNDEEP
amazon-web-servicesamazon-s3amazon-athena
Maxx
Maxx

Reputation: 846

How to use multiple file format in Athena

I have multiple file with different formats (csv, json and parquet) in s3 bucket directory (All files are in same directory). All files have same structure. How can I use these files to create Athena table?

Do we have provision to provide different Serde while creating table?

Edit: Table gets created but there is no data when I preview table.

Upvotes: 1

Views: 1373

Answers (1)

jbgorski
jbgorski

Reputation: 1939

There are a few options, but in my opinion it is best to create the separate paths (folders) for each type of files and run Glue Crawler on each of them. You will have multiple tables, but you can consolidate them by using Athena views or you can convert these files to one format by using Glue (for instance).

If you want to have the files in one folder you can use include and exclude patterns in Glue Crawler. Also in this case you will have to create seperate table for each type of file.

https://docs.aws.amazon.com/glue/latest/dg/add-crawler.html

Upvotes: 1

Related Questions