CODEWITHSUNDEEP
azureparquetazure-data-lake-gen2azure-synapse
Larry F
Larry F

Reputation: 11

Copy parquet file from Azure data lake storage account to Synapse data warehouse table failed

I used COPY INTO statement to copy csv file in ADLS Gen2 to Synapse table successfully with share access signature as credential. However, when I try to copy snappy.parquet file in the same storage account (different container) into a table in the same data warehouse, got error: "Error occurred while accessing HDFS: Java exception raised on call to HdfsBridge_Connect. Java exception message: Configuration property mystorage.dfs.core.windows.net not found.". my code is:

CREATE EXTERNAL FILE FORMAT pqt  
WITH (  
    FORMAT_TYPE = PARQUET  
    ,DATA_COMPRESSION = 'org.apache.hadoop.io.compress.SnappyCodec'               
    );  
COPY INTO [dbo].table
FROM 'https://mystorage.dfs.core.windows.net/../*.parquet'
WITH
(
  FILE_FORMAT =pqt
  ,CREDENTIAL=(IDENTITY= 'Shared Access Signature', SECRET='sas token') )

Do you know how to solve this issue?

Thanks

Upvotes: 1

Views: 1337

Answers (1)

HarithaMaddi-MSFT
HarithaMaddi-MSFT

Reputation: 551

Can you please try using blob in URL of location instead of dfs as this seems working for me even though I am referring to ADLS Gen 2 when loading Parquet file as below.

enter image description here

Upvotes: 0

Related Questions