CODEWITHSUNDEEP
pythoninfinite-loop
Zdenko Brzic
Zdenko Brzic

Reputation: 1

python endless loop stops writing to file

I am downloading dynamic data from api server and writing it to file by means of an endless loop in python. For whatever reason the program stops writing to file after couple thousand lines, while the program seems to be running. I am not sure where the problem is. The program does not give error, ie. it is not that the API server is refusing response. When restarted it continues as planned. I am using python 3.6 and win10.

Simplified version of the code looks something like:

import requests, json, time
while True:
    try:
        r = requests.get('https://someapiserver.com/data/')
        line = r.json()

        with open('file.txt', 'a') as f:
            f.write(line)
        time.sleep(5)
    except:
        print('error')
        time.sleep(10)

Upvotes: 0

Views: 423

Answers (1)

Petar Vasilev
Petar Vasilev

Reputation: 281

Try opening the file first and keeping the lock on it like so:

import requests, json, time

f = open('file.txt', 'a')

while True:
    try:
        r = requests.get('https://someapiserver.com/data/')
        line = r.json()
        f.write(line)
        f.flush()
        time.sleep(5)
    except:
        print('error')
        time.sleep(10)

f.close() # remember to close the file

The solution is ugly but it will do.

Upvotes: 1

Related Questions