CODEWITHSUNDEEP
amazon-web-servicesprestoamazon-athena
Sunil Kumbhar
Sunil Kumbhar

Reputation: 175

How does AWS Athena manage to execute queries immediately?

Does Athena have a gigantic cluster of machines ready to take queries from users and run them against their data? Are they using a specific open-source cluster management software for this?

Upvotes: 1

Views: 894

Answers (1)

Sayat Satybald
Sayat Satybald

Reputation: 6580

I believe AWS will never disclose how they operate Athena service. However, as Athena is managed PrestoDB the overall design can be deduced based on that.

PrestoDB does not require cluster manager like YARN, Messos. It has own planner and scheduler that is able to run SQL physical plan on worker nodes.

I assume that AWS within each availability zone maintains PrestoDB coordinator connected to data catalog(AWS Glue) and set of presto worker. Workers are elastic and autoscaled. In case of inactivity, they're downscaled, but when the burst of activity occurs new workers added to the cluster.

Upvotes: 5

Related Questions