CODEWITHSUNDEEP
azure-functionssnowflake-cloud-data-platform
Kanike
Kanike

Reputation: 33

Snowflake External Function - Parallel/Concurrent

I have a question regarding parallelism of snowflake external function. Here is my understanding:

  1. external function sends a set of rows to be processed to a remote service (in my case azure function httpTrigger).
  2. remote service handles those rows and sends output - one output for each row.

My question is does external function send multiple httpTrigger requests asynchronously sending a set of rows for each request. Is concurrency pre-built in the snowflake external function request? I know that concurrency in Azure functions to scale. Could you please guide me.

Upvotes: 2

Views: 403

Answers (1)

Felipe Hoffa
Felipe Hoffa

Reputation: 59165

From the docs:

Furthermore, the concurrency estimate should take into account that Snowflake can parallelize external function calls. A single query from a single user might cause multiple calls to the remote service in parallel. Several factors affect the number of concurrent calls from Snowflake to a proxy service or remote service, including:

https://docs.snowflake.com/en/sql-reference/external-functions-implementation.html#concurrency

Upvotes: 1

Related Questions