How to read a parquet file in Azure Databricks?

Question

I have few parquet files stored in my storage account, which I am trying to read using the below code. However it fails with error as incorrect syntax. Can someone suggest to me as whats the correct way to read parquet files using azure databricks?

val data = spark.read.parquet("abfss://containername@storagename.dfs.core.windows.net/TestFolder/XYZ/part-00000-1cf0cf7b-6c9f-41-a268-be-c000.snappy.parquet")
display(data)

Answer 1

abfss://containername@storagename.dfs.core.windows.net/TestFolder/XYZ/part-00000-1cf0cf7b-6c9f-41-a268-be-c000.snappy.parquet

As per the above abfss URL you can use delta or parquet format in the storage account.

Note: If you created delta table, part file creates automatically like this part-00000-1cf0cf7b-6c9f-41-a268-be-c000.snappy.parquet.As per above code it is not possible to read parquet file in delta format .

I have written the datafram df1 and overwrite into a storage account with parquet format.

df1.coalesce(1).write.format('parquet').mode("overwrite").save("abfss://<container>@<stoarge_account>.dfs.core.windows.net/<folder>/<sub_folder>")

Scala

val df11 = spark.read.format("parquet").load("abfss://<container>@<stoarge_account>.dfs.core.windows.net/demo/d121/part-00000-tid-2397072542034942773-def47888-c000.snappy.parquet")
display(df11)

python

df11 = spark.read.format("parquet").load("abfss://<container>@<stoarge_account>.dfs.core.windows.net/demo/d121/part-00000-tid-2397072542034942773-def47888-c000.snappy.parquet")
display(df11)

Output: