CODEWITHSUNDEEP
snowflake-cloud-data-platform
Nagendra Sharma
Nagendra Sharma

Reputation: 31

Can a notification be sent when Snowpipe fails to copy files in Snowflake?

I have created a Snowflake external stage on AWS S3 bucket. Also, I have setup a Snowpipe which executes the copy command when there is a file PUT on the S3 bucket. The Snowpipe gets notification from an SNS notification. The pipe has been configured to abort on error. Now, if there is any error while copying files from S3 into Snowflake ...say due to data mismatch... the Snowpipe simply aborts. Is there a way I get notified when the pipe fails? Because, I do not know if I am loosing any file from getting copied due to error.

Upvotes: 3

Views: 3093

Answers (3)

user19233080
user19233080

Reputation: 1

Just try unloading reject records to S3 and configure a notification in S3.

Upvotes: 0

Sriga
Sriga

Reputation: 1321

No, Currently snow pipe won't send any notification on failure. Its recommended to review the file status in snow pipe queue or use any third part tools to monitor your snowpipe queue.

Upvotes: 1

Mike Walton
Mike Walton

Reputation: 7339

Snowflake does not currently have notification/alerting services available for anything other than RESOURCE MONITORs, which are for warehouse credit consumption. The best way to monitor issues with Snowpipe is to leverage a third party reporting/monitoring tool that queries the Snowflake views.

I would recommend reviewing the following github project, as well, which is an open-source monitoring/alerting tool written for Snowflake:

https://github.com/snowflakedb/SnowAlert

Upvotes: 1

Related Questions