Why can't I delete a directory created in ProgramData?

Question

I' trying to create a directory and then delete it (for testing purposes, which I will ommit, but can give details if needed).

Like this:

>>> import os
>>> os.makedirs('C:\\ProgramData\\dir\\test')
>>> os.remove('C:\\ProgramData\\dir\\test')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
PermissionError: [WinError 5] Access is denied: 'C:\\ProgramData\\dir\\test'

I always get access denied, although I'm running the interpreter as an admin. Also I have no problem manually deleting the directory.

Answer 1

Use os.rmdir to remove a directory. On Windows this is implemented by calling the WinAPI function RemoveDirectory. os.remove is implemented by calling DeleteFile, which is only meant for deleting files. If the filename argument is a directory, the call fails and it sets the last error code to ERROR_ACCESS_DENIED, for which Python 3 raises a PermissionError.

In this case the access denied error is based on the NTSTATUS code STATUS_FILE_IS_A_DIRECTORY, i.e. RtlNtStatusToDosError(0xC00000BA) == 5. Often the kernel's status code is more informative than the corresponding WinAPI error, but not always, and there isn't always a simple mapping from one to the other, depending on the division of labor between the kernel, system processes, services, environment subsystems, and the application. In this case I think the kernel status code is undeniably more informative than a generic access denied error.

At a lower level the cause of the error when trying to delete a directory via DeleteFile is that it calls the system service NtOpenFile with the FILE_NON_DIRECTORY_FILE flag set in OpenOptions, whereas RemoveDirectory specifies FILE_DIRECTORY_FILE. Subsequently both functions call NtSetInformationFile to set the FileDispositionInformation to delete the file or directory.

---

Just to be a contrarian, let's implement the entire sequence using only file operations on an NTFS file system.

>>> import os, pathlib
>>> base = pathlib.Path(os.environ['ProgramData'])

Create the 'dir' directory:

>>> dirp = base / 'dir::$INDEX_ALLOCATION'
>>> open(str(dirp), 'w').close()
>>> os.path.isdir(str(dirp))
True

By manually specifying the stream type as $INDEX_ALLOCATION, opening this 'file' actually creates an NTFS directory. Incidentally, you can also add multiple named $DATA streams to a directory. Refer to the file streams topic.

Next create the 'test' subdirectory and call os.remove to delete it:

>>> test = base / 'dir' / 'test::$INDEX_ALLOCATION'
>>> open(str(test), 'w').close()
>>> os.path.isdir(str(test))                       
True
>>> os.remove(str(test))
>>> os.path.exists(str(test))
False

You may be surprised that this worked. Remember the filename in this case explicitly specifies the $INDEX_ALLOCATION stream. This overrules the FILE_NON_DIRECTORY_FILE flag. You get what you ask for. But don't rely on this since these streams are an implementation detail of NTFS, which isn't the only file system in use on Windows.

Answer 2

Use os.rmdir to delete a folder. os.remove is for deleting files.