CODEWITHSUNDEEP
snowflake-cloud-data-platformazure-data-factoryazure-sql-managed-instance
Dan
Dan

Reputation: 141

Azure Data Factory Managed Instance -> Snowflake text with escape characters

I've got a Copy Data Activity in Data Factory which takes a table from a SQL Server Managed Instance and puts it into a Snowflake Instance. The activity uses a temporary staging BLOB account.

When debugging the pipeline it's failing. The error comes up as "Found character 't' instead of record delimiter '\r\n'". It looks like it's caused by escape characters, but there is no options available to deal with escape characters on a temporary stage.

I think I could fix this by having two activities 1 moving Managed Instance to BLOB and 1 moving BLOB to Snowflake, but would prefer to handle it with just the 1 if possible.

I have tried to add to the user properties;

                    {
                        "name": "escapeQuoteEscaping",
                        "value": "true"
                    }

Is there anything else I could add in here?

Thanks, Dan

Upvotes: 3

Views: 323

Answers (1)

Robert Long
Robert Long

Reputation: 6877

It's the file format where you specify the details of the file being ingested, not the stage.

There are many options including the specification of delimiters and special characters within the data. The message

Found character 't' instead of record delimiter

suggests that you may have a tab-delimited file, so you could set \t as the delimiter in the file format.

https://docs.snowflake.com/en/sql-reference/sql/create-file-format.html

Upvotes: 0

Related Questions