CODEWITHSUNDEEP
pythonpython-3.xpermission-denied
D W
D W

Reputation: 99

Permission denied error on filewrite function

I wrote some code to parse XML and output the parsed information to two files. I then read those files and submit to a site with requests POST module. When I try to append the response from said POST to a file, I've been receiving PermissionError, [Errno 13] Permission Denied error (from line 112, submittoSite function in the code posted below).

The section it's happening is thus:

with open(xmfp + "nsresponse.csv", "a") as responsefile:
    responsefile.write(r.text + ", %d" % r.status_code + "\n")

Why is the file giving me a permission denied error, when I'm only writing it directly in line with the function? I write to the files that I'm parsing the XML out to fine, and this doesn't happen. It's at the end of a function. It's not trying to do more than one at a time, like threading, and doesn't a 'with' statement close the file once it's done?

Before saying something like 'write it to an array and then write it all to a file at once', I'm specifically not doing that because I'm doing 250k'ish POST calls (one of the reasons I wanted to multithread it) and I'm trying to keep RAM down.

Here's the full code (slightly edited due to work things)

NOTE: If you've read the code and are wondering about the threading stuff, I realize the multithreading stuff in there is extraneous now, but I'm still hoping that at some point I'll figure out how to get that to run without them interfering with each other, even if I have to rewrite stuff, so I'm just leaving it in there while I work on this.

Upvotes: 0

Views: 2200

Answers (2)

T.Woody
T.Woody

Reputation: 1218

Split up the code into several blocks to see where the error is coming from. Let your code snippet that you originally provided look like this instead for testing the error:

with open(xmfp + "nsresponse.csv", "a") as responsefile:
    foo = r.text + ", %d" % r.status_code + "\n"
    print(foo)
    responsefile.write("bar")

Furthermore, it appears after looking at the full code, the issue appears to be somewhere else. Test case:

import requests
import json
w = 'http://google.com'
r = requests.post(w)
foo = r.text
bar = r.status_code
baz = foo + ", %d" % bar + "\n"

with open('out.temp', 'a') as f:
 f.write(baz)

Upvotes: 1

blhsing
blhsing

Reputation: 106533

Windows does not allow opening of a file with write access if it is already opened by another thread or process with write access, so if two of your threads are trying to write to the file at the same time, one of them will get permission denied.

You should instead place a lock before trying to open the file with write permission:

from threading import Lock
def worker(lock):
    with lock:
        with open(xmfp + "nsresponse.csv", "a") as responsefile:
            responsefile.write(r.text + ", %d" % r.status_code + "\n")
if __name__ == '__main__':
    lock = threading.Lock()
    with ThreadPoolExecutor(max_workers=1) as executor:
        future = executor.submit(worker, lock)

Upvotes: 0

Related Questions