Reputation: 77
I understand that I can set the number or size of files using "Bucketing" method (Refer to this guide: https://aws.amazon.com/premiumsupport/knowledge-center/set-file-number-size-ctas-athena/ )
I also known that I can set the number of output file by using Glue job repartition. However, what I want to confirm is that: Am I right if I understand that Bucketing is the only way so that the number of output file can be set if I use Athena? Is there any other methods?
Upvotes: 2
Views: 942
Reputation: 77
I have search for many sources in the Internet and confirm with AWS Support team that Bucketing is the only way to control the number of output files.
Thank you guys for your help!
Upvotes: 2
Reputation: 132862
There are currently no other methods to control the number of output files of a CTAS operation.
Upvotes: 3