CODEWITHSUNDEEP
pythonlinuxfile-iopython-asyncio
Enfors
Enfors

Reputation: 960

How to read a file as it is being written in real time with Python 3 and asyncio, like "tail -f"

I want to write a Python program on Linux that reads a log file in real time as it is being written, for the purpose of sending an alarm if it detects certain things in the log. I want this to use asyncio for several reasons - I'm trying to build a framework that does many things at the same time based on asyncio, and I need the practice.

Since I'm using asyncio, I obviously don't want to use a blocking read to wait at the end of the input file for more lines to be written to it. I suspect I'll have to end up using select, but I'm not sure.

I suspect that this is pretty simple, but I have a hard time finding an example of how to do this, or coming up with one of my own even though I've dabbled a little bit in asyncio before. I can read and mostly understand other asyncio examples I find, but for some reason I find it difficult to write asyncio code of my own.

Therefore, I'd be very grateful if someone could point me to an example. Bonus points if the same technique also works for reading from stdin rather than a file.

Upvotes: 1

Views: 2375

Answers (1)

user4815162342
user4815162342

Reputation: 155046

I suspect I'll have to end up using select, but I'm not sure. I suspect that this is pretty simple, but I have a hard time finding an example of how to do this

With asyncio, the idea is that you don't need to select() yourself because asyncio selects for you - after all, a select() or equivalent is at the heart of every event loop. Asyncio provides abstractions like streams that implement a coroutine facade over the async programming model. There are also the lower-level methods that allow you to hook into select() yourself, but normally you should work with streams.

In case of tail -f, you can't use select() because regular files are always readable. When there is no data, you get an EOF and are expected to try again later. This is why tail -f historically used reads with pauses, with the option to deploy notification APIs like inotify where available.

Upvotes: 1

Related Questions