CODEWITHSUNDEEP
pysparkazure-sql-databaseazure-synapse
ZZZSharePoint
ZZZSharePoint

Reputation: 1351

How Do i read the Lake database in Azure Synapse in a PySpark notebook

Hi I created a Database in Azure Synapse Studio and I can see the database and table in there, Now I have created a Notebook where I have added the required libraries but I am unable to read the table by below code. Can anyone fix what wrong am i doing here ?

My database name is Utilities_66_Demo . It gives me error as

AnalysisException: Path does not exist: abfss://[email protected]/Utilities_66_Demo.parquet

From where should I take the path? I tried to follow the MS article. Where Do I read path? if I click on edit Database, i get this

enter image description here

%%pyspark
df = spark.read.load('abfss://[email protected]/Utilities_66_Demo.parquet', format='parquet')
display(df.limit(10))

Trying to access the created Lake Database Table: Selected Azure Synapse Analytics:

enter image description here

I select my workspace and in dropdown there is no table shown:

enter image description here

I select Edit and put my Db name and Table name and it says Invalid details.

Now I select Azure Dedicated Synapse Pool from Linked Service,

I get no option to select in SQL Pool or Table, and without SQL Pool I am unable to create a Linked service just by inserting Table name:

enter image description here

Upvotes: 1

Views: 2729

Answers (1)

AnnuKumari
AnnuKumari

Reputation: 563

You can directly go to your ADLS and right click the parquet file and select properties. There, you will be able to find the ABFSS path which is in the format : abfss://<container_name>@<storage_account_name>.dfs.core.windows.net/<path

enter image description here

Upvotes: 1

Related Questions