CODEWITHSUNDEEP
snowflake-cloud-data-platform
ajcoder
ajcoder

Reputation: 247

Choosing a warehouse size for Tasks

I came across this question and was puzzled. How to determine the size of virtual warehouse used for a task?

A. Root task may be executed concurrently (i.e multiple instances), it is recommended to leave some margins in the execution window to avoid missing instances of execution.

B. Querying (select) the size of the stream content would help determine the warehouse size. For ex, if querying large stream content, use a larger warehouse size.

C. If using stored procedure to execute multiple SQL statements, it's best to test run the stored procedure separately to size the compute resource first.

D. Since task infrastructure is based on running the task body on a schedule, it's recommended to configure the virtual warehouse for automatic concurrency handling using Multicluster warehouse to match the task schedule.

Upvotes: 0

Views: 1041

Answers (1)

Felipe Hoffa
Felipe Hoffa

Reputation: 59325

Check the new "serverless" Snowflake tasks:

In this case, Snowflake will automatically determine what's the best warehouse size.

You can give a hint to Snowflake on what size to start with, using USER_TASK_MANAGED_INITIAL_WAREHOUSE_SIZE.

Specifies the size of the compute resources to provision for the first run of the task, before a task history is available for Snowflake to determine an ideal size. Once a task has successfully completed a few runs, Snowflake ignores this parameter setting. https://docs.snowflake.com/en/sql-reference/sql/create-task.html

The implications on billing are described here:

Upvotes: 1

Related Questions