CODEWITHSUNDEEP
hadoop-yarnsnowflake-cloud-data-platform
Micah Pearce
Micah Pearce

Reputation: 1945

What is the Amazon Snowflake execution engine

When you are on Hadoop, you can have YARN manage the Hadoop jobs, resources, etc ...

What is the equivalent form for Amazon's Snowflake?

Hadoop (HDFS) is to YARN as Snowflake is to __________

Upvotes: 1

Views: 313

Answers (2)

Marcin Zukowski
Marcin Zukowski

Reputation: 4729

You should read the technical paper Snowflake team published in SIGMOD 2016.

It describes the architecture in quite some detail.

Upvotes: 5

dbaOnTap
dbaOnTap

Reputation: 186

Interesting question here. To level set, Snowflake is a Data Warehouse SaaS offering that is deployed on both AWS and Azure, I wouldn't categorize it as "Amazon's Snowflake".

Snowflake does not have a separate tool managing resources for the data warehouse clusters. It is built into it's 3 tier architecture. This link will provide some details on that: https://www.snowflake.com/product/architecture/

As Snowflake is a cloud native data warehouse the inherent elasticity of the cloud does give administrators the ability to dynamically scale compute resources both vertically and horizontally to address things such as queuing and parallelism.

Hope this helps.

Upvotes: 4

Related Questions