CODEWITHSUNDEEP
sql-serversnowflake-cloud-data-platform
Nick.Maco
Nick.Maco

Reputation: 1

Migrating SQlserver NVARCHAR(MAX) --> Snowflake

Looking for some past experience from others on this. Scenario : We have SQLserver 2016 environment with tables defined with NVARCHAR(MAX) as the datatype with many records getting very close to the 2GB limit allowed by Sqlserver.

We would like to migrate this data to Snowflake but with VARCHAR limited to 16MB there doesn't seem to be any other suitable data type.

The data migration needs to retain the complete data set, so any truncation of data is not acceptable.

Running a transformation to divide the 2GB of data into 16MB chunks is not feasible either as that would generate 100+ columns on the snowflake end.

Let me know what others have done in the situation.

Regards Nick

Upvotes: 0

Views: 1788

Answers (1)

Tom Meacham
Tom Meacham

Reputation: 191

I would take a look at Snowflake's unstructured file support. As others have noted, a 2GB varchar sounds a lot like a file.

https://docs.snowflake.com/en/user-guide/unstructured-intro.html

This method supports files of unlimited size (so you no longer have to worry about the file size approaching the limit) and gives users the ability to download the files as needed by providing a URL (including time-limited URLs).

This can be used for any file type including video, audio, pdfs, etc.

Upvotes: 1

Related Questions