CODEWITHSUNDEEP
azure-synapseazure-data-lake-gen2azure-synapse-link
Despicable me
Despicable me

Reputation: 700

Permission denied error while accessing files under mount point via local file system API in Azure Synapse Notebook

Working from Azure Synapse Notebook, I have mounted the ADLS Gen2 folder using LinkedServiceA as per below command,

mssparkutils.fs.mount(
 "abfss://<CONTAINER>@<STORAGENAME>.dfs.core.windows.net/", #ADLS GEN 2 PATH
 "/adlsmount", #Mount Point Name
 { "linkedService" : "<REPLACE LINKED SERVICE NAME>"})

I am trying to access the mount path using the local file system API as below.

Folder structure is like container/team/SALES/BILLING/YEAR.

LinkedServiceA that is used to create the mount is having access to only SALES and its subfolders.

import os

jobId = mssparkutils.env.getJobId()

synfs_bill_path = f'synfs:/{jobId}/adlsmount/team/SALES/BILLING/YEAR' #SYNFS
local_bill_path=  f'/synfs/{jobId}/adlsmount/team/SALES/BILLING/YEAR' #Local File System

mssparkutils.fs.ls(synfs_bill_path) #this is working
bills = os.listdir(local_bill_path) #this is failing with permission denied error

But i am able to list all the parent directories using Local File System API Path using os lib

local_base_path=  f'/synfs/{jobId}/adlsmount/team/SALES/'
bills = os.listdir(local_base_path) #this is working, 
print(bills ) #lists "BILLING" folder

Error Message

PermissionError: [Errno 13] Permission denied: '/synfs/152/adlsmount/team/SALES/BILLING/YEAR'
Traceback (most recent call last):

Spark API using synfs_bill_path is also working. I wanted to process large number of small files in the SALES/BILLING/YEAR to reduce the number of files.(spark read fails with large number of files)

Upvotes: 0

Views: 1593

Answers (1)

NiharikaMoola
NiharikaMoola

Reputation: 5074

I have tried to repro your code in my lab environment and your code works fine without any errors for me.

enter image description here

Permission denied [Errno 13] is mostly seen when you try to access a path without having the necessary permissions. Please make sure the user has all the necessary permissions.

Upvotes: 0

Related Questions