CODEWITHSUNDEEP
pythonpython-3.xfileupdating
Rock Blazer
Rock Blazer

Reputation: 11

Appropriate way to read a constantly updating (20 times per second) file in Python 3?

I'm working with a csv file that's constantly growing, with about 20 lines being added per second. Each line needs to be parsed. The code snippet I have below does work, but it seems to stop updating after a bit. It's running in its own thread and if I manually update the csv file (ie. a new line every few seconds), it seems to work perfectly fine.

file=open('data.csv', 'r')
while True:
    line=file.readline()
    if len(line) > 2:
        print(line)
        #parse

This is on Ubuntu 14.04 and Python 3.5 (unfortunately, I'm stuck with these versions). Strangely enough, I haven't noticed any issues when running on Windows 7. Is there a better way to approach this?

Upvotes: 1

Views: 1452

Answers (1)

duong_dajgja
duong_dajgja

Reputation: 4276

Since OP has claimed that the file is appended then I would suggest to try answers to How can I tail a log file in Python?. In short, you could do:

  • Your python script should use the built-in tail and check return from the tail.

  • Use an external python module named sh

Upvotes: 1

Related Questions